Hacettepe Üniversitesi Eğitim Fakültesi Dergisi (H. U. Journal of Education) ??: SS-SS [201?] doi: 10.16986/HUJE.2018046755 [Araştırma Makalesi / Research Article] The Effect of Flipped Learning Approach on Academic Achievement: A Meta-Analysis Study* Ters-Yüz Edilmiş Öğrenme Yaklaşımının Akademik Başarıya Etkisi: Bir Meta-Analiz Çalışması İbrahim KARAGÖL**, Emrullah ESEN***  Received: 11.06.2018  Accepted: 21.12.2018  Published: 24.12.2018 Kaynakça Bilgisi: Karagöl, İ., & Esen, E. (2018). Ters-yüz edilmiş öğrenme yaklaşımının akademik başarıya etkisi: Bir meta-analiz çalışması. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi. Advance online publication. doi: 10.16986/HUJE.2018046755 Citation Information: Karagöl, İ., & Esen, E. (2018). The effect of flipped learning approach on academic achievement: A meta-analysis study. Hacettepe University Journal of Education. Advance online publication. doi: 10.16986/HUJE.2018046755 ABSTRACT: This study aims to examine the effect of the flipped learning approach on academic achievement through meta-analysis method. The study consists of the published articles in scientific journals, master's and doctoral theses which have the necessary statistical data. The studies were obtained from various databases such as “EBSCOhost, ProQuest, JSTOR, Google Scholar, Turkish Academic Network and Information Center (TUBITAK ULAKBIM) Social Sciences Database, Turkish Council of Higher Education National Thesis Center and ERIC” by using keywords such as “flipped classroom”, “flipped learning”, “academic achievement”. Within the scope of inclusion criteria, 55 studies were obtained. The data set was determined as 80 since the effect of the flipped learning approach on academic achievement was analysed separately for each course in some studies. The results of the study reveal that there is a positive effect of the flipped learning approach on academic achievement compared to traditional learning approach. Besides, there is not a significant difference according to the implementation period. The effect of flipped learning on academic achievement is higher in small groups, and there is a significant difference between the groups according to being national/international. Keywords: Flipped classroom, flipped learning, meta-analysis, academic achievement ÖZ: Bu araştırmada ters-yüz edilmiş öğrenme yaklaşımının akademik başarıya etkisinin meta-analiz yöntemiyle incelemesi amaçlanmıştır. Bu amaç doğrultusunda “ters-yüz edilmiş öğretim”, “ters-yüz edilmiş öğrenme”, “akademik başarı” gibi anahtar kelimeler kullanılarak EBSCOhost, ProQuest, JSTOR, Google Akademik, TÜBİTAK ULAKBİM Sosyal Bilimler Veri Tabanı, YÖK Tez ve ERIC arama motorları aracılığıyla ilgili tez ve makaleler elde edilmiştir. Dahil edilme ölçütleri kapsamında 55 çalışmaya ulaşılmıştır. Bazı çalışmalarda ters-yüz edilmiş öğrenme yaklaşımının akademik başarıya etkisi her bir ders için ayrı ayrı hesaplandığı için veri seti 80 olarak belirlenmiştir. Araştırmanın sonuçlarına göre, ters-yüz edilmiş öğrenme yaklaşımının akademik başarı üzerinde pozitif yönde bir etkisi olduğu; ters-yüz edilmiş öğrenme yaklaşımının akademik başarı üzerindeki etkisinin uygulama süresine göre değişmediği; ters-yüz edilmiş öğrenme yaklaşımının akademik başarı üzerindeki etkisinin küçük gruplarda daha yüksek olduğu ve çalışmaların ulusal/uluslararası olma durumuna göre gruplar arasında anlamlı bir farklılık olduğu sonuçlarına ulaşılmıştır. Anahtar Sözcükler: Ters-yüz edilmiş sınıf, ters-yüz edilmiş öğrenme, meta-analiz, akademik başarı * This study was presented orally at 5th International Eurasian Educational Research Congress between 2-5 of May, 2018 in Antalya. ** Res. Assist., Anadolu University, Faculty of Education, Department of Educational Sciences, Division of Curriculum and Instruction, Eskişehir-TURKEY. e-mail: karagolibrahim@gmail.com (ORCID: 0000-0002-70057710) *** Res. Assist., Anadolu University, Faculty of Education, Department of Educational Sciences, Division of Curriculum and Instruction, Eskişehir-TURKEY. e-mail: emrullahesen@anadolu.edu.tr (ORCID: 0000-0002-73014986) e-ISSN: 2536-4758 http://www.efdergi.hacettepe.edu.tr/ 2 İbrahim Karagöl, Emrullah Esen 1. INTRODUCTION Today, with the scientific and technological advancement, teaching and learning environments have started to change and evolve. Notably, the increasing use of smart devices and the internet have accelerated the integration of multi-media tools in educational environments. In line with technology and science, changing needs of learners, differentiation in instructional designs and growing opportunities form a basis for new teaching approaches to putting into practice. Flipped learning (or inverted), one of these approaches, has emerged as a new alternative to the traditional learning environment. Flipped learning is defined as a pedagogical approach that direct instruction moves out of the class via technology and internet (e.g. videos, podcasts, online blogs or available online materials.) while in-class time includes practice and collaborative activities which promote active learning (Abeysekera & Dawson, 2015; Arnold-Garza, 2014; Bergmans & Sams, 2012; Bishop & Vergler, 2013; Enfield, 2013). Similarly, Lage, Platt & Treglia (2000) defines flipped learning as “inverting the classroom means that have traditionally taken place inside the classroom now take place outside the classroom and vice versa” (p. 32). The origin of the flipped learning refers to Jonathan Bergmann & Aaron Sams, two chemistry teachers from Colorado, who used recorded lectures to provide instruction to secondary students who were missing classes (Bergman & Sams, 2012). There is no single form for flipped learning, but the term generally refers to a class design which provides pre-recorded lectures followed by in-class practice as a standard model that student gets exposed to five to seven minutes of lectures and do online quizzes and activities to test themselves before coming to the class (Educause, 2012). By moving the lecture time out of the class, class-time is freed for hands-on learning, individualised instruction, group collaboration and a remedial session with the teacher (Webb & Doman, 2016). Students work on activities and put their knowledge into practice (Salimi & Yousefzadeh, 2015). In the flipped learning approach, teachers might use “just-in-time teaching” to adjust their instruction according to web-based quizzes and questions that are done by students before class (Berett, 2012). With all these features, teachers and students collaborate in order to master the topic, concepts, and other areas of learner weakness (Harris, Harris, Reed & Zelihic, 2016). The flipped learning model is generally categorised under the concept of hybrid or blended learning, which utilises problem-based and active learning techniques and new technologies to engage students (Arnold-Gaza, 2014). For example, Staker & Horn (2012) categorise flipped learning model under rotation models in blended learning models. According to the flipped learning rotation model, “in a course or a subject (e.g. Math), students rotate on a fixed schedule between face-to-face teacher-guided practice on a school day and online delivery of content of the same subject from a remote location (often home) after school.” (p, 10). Teaching in flipped learning approach requires students to make pre-class preparation by watching recorded lectures, while in-class time spared for discussion, problem-solving activities and group-based activities related to the topic (Tune, Sturek & Basil, 2013; Pierce & Fox, 2012). The activities, such as group-based and problem-based activities, used in flipped classroom model are generally related to active learning which is considered to stem from constructivism (Abeysekera & Dawson, 2015; Arnold-Gaza, 2014; Bishop & Vergleher, 2013). It uniquely combines two incompatible learning theories by adopting active and problem-based learning activities founded on constructivism and instructional lectures which stem from the direct instruction method founded on the behaviourist approach (Bishop & Vergleher, 2013). Beside inverting instruction time and instruction tools, the flipped learning approach has changed the role of teachers and students (Educause, 2012; Harris et al., 2016). In a traditional classroom setting, teachers are considered as “sage on the stage” (Baker, 2000) who transmit knowledge and students are passive listeners whereas in a flipped classroom teacher serve as e-ISSN: 2536-4758 http://www.efdergi.hacettepe.edu.tr/ The Effect of Flipped Learning Approach on Academic Achievement: A Meta-Analysis Study 3 coaches, guides on the sides or subject matter experts contributing the learning process by collaborating with students (Educause, 2012; Harris et. al. 2016). In a traditional classroom environment, students must follow what is being said by the teacher (Zhonggen & Wang, 2016). They focus on the delivery of the teacher and cannot stop reflect upon what the teacher says and may miss the significant points while trying to transcribe the words (Educause, 2012). According to Sankoff (2014) traditional lectures often lead waste of precious resources and fail to make use of instructors’ experience, knowledge, and abilities by making him deliver the same information to the different groups. In such traditional environments, students often shy away from speaking up and asking for clarification while teachers are expected to fill students with knowledge (Harris et al., 2016). Other from that, teachers can not explore the deficiencies of students until the assignment is handed out, or the assessment is done (Dove & Dove, 2015). On the other hand, flipped learning approach is reported to have many advantages for students and teachers in the literature. Fulton (2012) listed advantages of flipped learning as; (1) students learn at their own pace and style, (2) allows efficient use of class time, (3) allows group discussion and peer instruction, (4) motivates teachers for professional development. Besides, as related studies suggest, flipped classroom gives more responsibility to students for learning (Educause, 2012), which increases in the awareness of metacognition (Yıldız & Kıyıcı, 2016). In addition, flipped learning approach is found to increase the motivation level of the students (Bhagat, Chang & Chang, 2016; Chao, Chen & Chuang, 2015), their self-efficacy (Kurt, 2017; Thai, Wever & Valcke, 2017), conceptual understanding (Olakanmi, 2017), attitudes towards learning (Chao, Chang & Chuang, 2015). In a flipped learning environment, the students turn into an active learner who participates in higher-level critical thinking, interactive and problemsolving activities and engage deep learning by using metacognition (Brame, 2013; Sharpe, 2016). For instance, in a study done by Tarazi (2016) students exposed to inverted teaching engaged in deep learning and showed a higher level of motivation. As another advantage, in a flipped context, students engage the lower level of cognitive work outside the classroom, and the higher level of cognitive work in the classroom (Sharpe, 2016), “where they have the support of the peers and instructors” (Brame, 2013, p. 1). Concerning academic achievement, the flipped learning approach and traditional learning approach have been subject to many types of research. Based on the post-test scores, while some research suggests there is a meaningful difference in favour of flipped learning (Aljeser, 2007; Aydın, 2016; Sickle, 2016; Sun & Wu, 2016; Turan, 2015; Webb & Doman, 2016; Salimi & Yousefzadeh, 2015; Zhonggen & Wang, 2016), some others say that there is no significant difference between flipped and traditional approach (Bishop, 2013; Brooks, 2014; Cashin, 2016; Clark, 2013; Crowford, 2017; Dixon, 2017; Faretta, 2016; Fraga & Harmon, 2017; Howell, 2013; Johnson, 2012; Montgomery, 2015; Overmyer, 2014; Saunders, 2014; Sharpe, 2016; Smith, 2016; Winter, 2013; Yavuz, 2016). In other words, all these studies suggest that there is no consensus on whether the flipped learning approach is significantly effective or not over the traditional approach. Either complex nature of social sciences and educational research or the presence of many threats, which are hard to remove and affect the internal validity of the experimental studies, might be the reasons for this contradiction (Üstün & Eryılmaz, 2014). Besides, Davies (2000) suggests that a single experiment has situation-specific limitations such as time, sample and context. In this regard, meta-analysis studies, which are utilised to bring together the findings of different research results on the same topic coherently and consistently to expand the sample and obtain reliable results, are seen crucial (Cohen, Manion & Morison, 2011; Dempfle, 2006; Petiti, 2000). In the literature review, there is only one meta-analysis study regarding the flipped learning approach and academic achievement (Hew & Lo, 2018). In their meta-analysis study, Hew & Lo (2018) focused on 28 studies, published between the years of 2012 and 2017, regarding the flipped learning approach and health care professional learners’ achievement. The e-ISSN: 2536-4758 http://www.efdergi.hacettepe.edu.tr/ İbrahim Karagöl, Emrullah Esen 4 researchers found out that the flipped learning approach has a significant effect on student achievement. Nevertheless, the study conducted by Hew and Lo (2018) differs from our study in terms of scope, moderator variables, and inclusion criteria. In their study, Hew and Lo (2018) included studies conducted with health professionals (medical students or learners) and examined 28 studies in terms of a) student initial equivalence, b) instructor equivalence, c) research design, d) types of students; e) pre-class component of flipped classroom and f) inclass component of flipped classroom. In our study, the moderator variables determined as; a) educational level, b) implementation period, c) sample size, d) practitioner and e) being national or international. For these reasons, it is considered a need to conduct a meta-analysis including national and international studies in order to reveal the effect of the flipped learning approach regarding the mentioned moderator variables on academic achievement. By bringing together the findings of the studies conducted with different groups, durations, implementors, places, and sample sizes, this study is expected to contribute to the literature by producing scientific proof concerning the effect of flipped learning approach on academic achievement. Consequently, the purpose of this research is to find out the effect of the flipped learning approach on students’ academic achievement in comparison to traditional learning approach by using meta-analysis method. Thus, the current study is seeking the answers to the following questions: 1) What is the effect of the flipped learning approach on students’ academic achievement? 2) Does the effect of the flipped learning approach on academic achievement differ according to educational level? 3) Does the effect of the flipped learning approach on academic achievement differ according to the implementation period? 4) Does the effect of the flipped learning approach on academic achievement differ according to sample size? 5) Does the effect of the flipped learning approach on academic achievement differ according to the practitioner? 6) Does the effect of the flipped learning approach on academic achievement differ according to being national/international? 2. METHOD 2.1. Research Model A meta-analysis method was used to examine the effect of the flipped learning approach on academic achievement in this study. Meta-analysis, which is the analysis of other analyses (Cohen et al., 2007), is a statistical method used to gather the findings of different researches on the same topic and to obtain more accurate and reliable results by expanding the sample (Dempfle, 2006; Petitti, 2000). 2.2. Data Collection The studies included in this meta-analysis consist of the published articles in scientific journals, master's and doctoral theses which have the necessary statistical data. The studies were obtained from various databases such as “EBSCOhost, ProQuest, JSTOR, Google Scholar, ERIC, Turkish Academic Network and Information Center (TUBITAK ULAKBIM) Social Sciences Database and Turkish Council of Higher Education National Thesis Center” by using keywords such as “flipped classroom, flipped learning, academic achievement” and their Turkish equivalents. First of all, the abstract sections of the studies were read, and the same copies were eliminated. After the first phase, 365 studies were taken to be analysed. Four thesis could not be reached due to the restrictions on them. Articles and thesis which are not suitable for the research problem (n=258), and do not meet the inclusion criteria (n=51) were eliminated. If the e-ISSN: 2536-4758 http://www.efdergi.hacettepe.edu.tr/ The Effect of Flipped Learning Approach on Academic Achievement: A Meta-Analysis Study 5 same study is published both as a thesis and an article, only the theses are taken into the scope of the study (n=1) since they contain more detailed information compared to the articles. Thus, there are 55 studies remained. The data set was determined as 80 since the effect of the flipped learning approach on academic achievement were analysed separately for each course in some studies. For instance, in their study, Salimi & Yousefzadeh (2015) analysed the effect of the flipped learning approach on English, Arabic, math, science and geography class separately. Therefore, there are five different effect sizes in a single study. A diagram showing the process of the literature review is shown below. Figure 1: Diagram showing the process of literature review 2.2.1. Inclusion criteria The following criteria were taken into consideration for the studies included in this study:  Studies should be the published articles in scientific journals, master's or doctoral theses which are written either in English or Turkish. e-ISSN: 2536-4758 http://www.efdergi.hacettepe.edu.tr/ İbrahim Karagöl, Emrullah Esen 6   Studies should be designed as experimental models. The participants in the experimental group should be applied flipped learning approach, whereas the participants in the control group should be applied traditional learning approach. Standard deviations, arithmetic means, and the number of participants regarding the control and the experimental groups should be identified.  2.3. Coding of Data In the process of coding data, a coding form was developed by the researchers in order to determine whether or not the studies conform to the meta-analysis inclusion criteria and to make comparisons between studies. The coding form includes the name of the study, the location of the study, application period of the study, practitioners who conducted the application, sample size, standard deviations, arithmetic means, and the number of the participants regarding the control and the experimental groups. Coding of data was carried out by the researchers independently. Besides, randomly selected 11 studies were asked to be coded by another research assistant who was continuing his PhD in educational sciences. The reliability coefficient among the coders was calculated by using Miles and Huberman's (1994) reliability formula. As a result of the calculations, the reliability coefficient between the first researcher and the research assistant was found as 93%, the reliability coefficient between the second researcher and the research assistant was found as 92% and the reliability coefficient between the researchers was found as 99%. According to Miles and Huberman’s (1994) reliability formula, the present study was reliable since the reliability coefficients were more than 70%. The content which the coders had not agreed on was then discussed till they reached an agreement. 2.4. Data Analysis In the study, the effect sizes of all studies and the common effect size were calculated using Comprehensive Meta-Analysis (CMA 2.2) statistical package program. Specific classifications are used when interpreting the effect sizes obtained from the meta-analysis. According to Cohen et al. (2007, p. 521), the classification of effect size is as follows: • 0–0.20 = weak effect • 0.21–0.50 = modest effect • 0.51–1.00 = moderate effect • = strong effect >1.00 Before calculating the effect sizes in the meta-analysis, it is decided whether to use a fixed effects model or random effects model. In the fixed effects model, it is assumed that all the factors that influence the effect size are the same in all studies (Borenstein, Hedges, Higgins & Rothstein, 2013). However, in the random effects model, it is assumed that the effect sizes differ from study to study (Ellis, 2010). Which of these models to be used in the meta-analysis is decided by determining whether the effect sizes are homogeneous or not (Pigott, 2012). If the effect sizes are distributed homogeneously, the fixed effects model is used; if the effect sizes are heterogeneously distributed, then the random effects model is used (Borenstein, Hedges, Higgins & Rothstein, 2009). e-ISSN: 2536-4758 http://www.efdergi.hacettepe.edu.tr/ The Effect of Flipped Learning Approach on Academic Achievement: A Meta-Analysis Study 7 3. FINDINGS 3.1. Findings Regarding the Research Question: “What is the effect of the flipped learning approach on students’ academic achievement?” In order to calculate the effect sizes of the studies, the effect model to be used should be determined. First, homogeneity of the studies should be tested with a fixed effects model. Findings regarding the homogeneity of the studies with fixed effects model and the overall effect size are given in Table 1 below. Table 1: Findings regarding the effect size according to the fixed effects model ES .406 (df) 79 X2 100.749 Qb 434.834 SE .029 Effect size at 95% Confidence Interval I2 ESMin ESMax 81.832 .350 .462 Homogeneity value of the studies included in the study was found as Q = 434.834 according to the fixed effects model. The critical value obtained from the chi-square table at 95% significant level with 79 degrees of freedom is 100.749. Q statistical value was found to exceed the critical value of the chi-square distribution with 79 degrees of freedom (for df=79, X2 (0.95) = 100.749). Accordingly, it shows that the study has a heterogeneous distribution. Similarly, a high I2 value indicates that the distribution is heterogeneous. Therefore, it is not possible that there is only one true effect underlying the value of effect size. In this case, the random effects model was preferred to be used in the study by the researchers. Table 2: Findings regarding the effect size according to the random effects model ES .566 n 80 SE .071 Z 8.017 p .000 Effect size at 95% Confidence Interval ESMin ESMax .428 .705 According to the results of the analysis through random effects model, the effect sizes of the studies were found as .428 for the lower limit of the 95% confidence interval, .705 for the upper limit of the 95% confidence interval and .566 for the average value of the effect size. According to Cohen et al. (2007), this value has a moderate effect. Besides, a positive value of the effect size indicates that the implication effect is in favour of the experimental group. Therefore, the effect of the flipped learning approach on academic achievement is more effective than the traditional learning approach. The forest plot of the studies demonstrating the distribution of effect size values calculated by the random effects model is shown in Figure 2. e-ISSN: 2536-4758 http://www.efdergi.hacettepe.edu.tr/ 8 İbrahim Karagöl, Emrullah Esen Figure 2: Forest plot demonstrating the distribution of effect size values In the forest plot, black squares show the effect size of the study and the horizontal lines passing through the squares indicate the confidence interval for the study. The longer the horizontal line, the larger the confidence interval. According to the forest plot, while the largest confidence interval belongs to Topalak (2016), the smallest belongs to Overmyer (2014). The other studies have similar values of weight percentages in this analysis. The diamond shape at the end of the plot shows the overall effect size for all studies. The vertical line at an effect size=0 shows the line of no effect (Akobeng, 2005; Ried, 2006). If the shape is not on the line of no effect, it means there is a significant difference between the two groups. If an effect size is found to be positive, it means that the performance is in favour of the experimental group (Wolf, 1986, p. 26). According to forest plot, the smallest effect size value is -2.314 and the highest effect size value is 7.824. While 67 studies have positive effect e-ISSN: 2536-4758 http://www.efdergi.hacettepe.edu.tr/ The Effect of Flipped Learning Approach on Academic Achievement: A Meta-Analysis Study 9 sizes, 13 studies have negative effect sizes. Consequently, the flipped learning approach implemented in 67 studies has a significant effect in favour of the experimental group. One of the most significant problems in meta-analysis studies is that the studies included in the meta-analysis may be biased (Sarıer, 2016). To demonstrate that there is no publication bias, it must be calculated that how many missing studies should be included in the analysis to make the effect size statistically insignificant (Borenstein et al., 2009). "Classic Fail-Safe N" analysis was conducted to investigate publication bias in the study. The results of the analysis are shown in Table 3 below. Table 3. Classic Fail-Safe N Analysis z value p value Alpha Z for alpha N p>number of missing studies 16.63 .00 .05 1.96 80 5783 As shown in Table 3, the p value is smaller than the alpha value, which is regarded as a demonstration that the study is reliable (Borenstein et al., 2013). According to the classic failsafe N analysis, the number of missing studies that would bring p value to >alpha=0.05 is 5783. Given a large number of studies, the analysis results are reliable and there is no publication bias. In addition to this analysis, it can be interpreted whether or not there is publication bias using the Funnel Plot given in Figure 3. Funnel Plot of Standard Error by Hedges's g 0,0 Standard Error 0,2 0,4 0,6 0,8 1,0 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 Hedges's g Figure 3: Funnel plot demonstrating the effect size of the studies Figure 3 shows the distribution of the studies included in the study. As the graph shows, the studies seem symmetrical, and there is no asymmetric distribution which means there is no publication bias. Because, in cases where there is publication bias, the distribution is asymmetric and skewed in the funnel graph (Üstün & Eryılmaz, 2014). e-ISSN: 2536-4758 http://www.efdergi.hacettepe.edu.tr/ İbrahim Karagöl, Emrullah Esen 10 3.2. Findings Regarding the Research Question: “Does the effect of the flipped learning approach on academic achievement differ according to educational level?” Findings regarding whether the effect size differs according to educational level are shown in Table 4. Table 4: Findings regarding the effect size according to educational level Variable QB p Educational Level Elementary School High School University 1.100 .577 n ES Effect Size at 95% Confidence Interval ESMin SE ESMax 15 21 44 .653 .448 .594 .345 .176 .401 .157 .139 .098 .961 .721 .786 It is found that critical value obtained from the chi-square table at 95% significant level with two degrees of freedom is 5.991. The homogeneity value among the groups according to educational level is (QB) 1.100. Since Q value is smaller than the critical value obtained from the chi-square table at 95% significant level with two degrees of freedom, there is not a statistically significant difference among the groups according to educational level. 3.3. Findings Regarding the Research Question: “Does the effect of the flipped learning approach on academic achievement differ according to the implementation period?” Findings regarding whether the effect size differs according to the implementation period are shown in Table 5. Table 5: Findings regarding the effect size according to the implementation period Variable QB p Implementation Period 1-4 weeks 5-8 weeks 9 or more weeks Unspecified 2.206 .531 n ES Effect Size at 95% Confidence Interval ESMin SE ESMax 14 38 20 8 .692 .582 .410 .707 .360 .371 .135 .268 .170 .108 .140 .224 1.024 .793 .685 1.146 It is found that critical value obtained from the chi-square table at 95% significant level with three degrees of freedom is 7.815. The homogeneity value among the groups according to the implementation period is (QB) 2.206. Since Q value is smaller than the critical value obtained from the chi-square table at 95% significant level with three degrees of freedom, there is not a statistically significant difference among the groups according to the implementation period. 3.4. Findings Regarding the Research Question: “Does the effect of the flipped learning approach on academic achievement differ according to sample size?” Findings regarding whether the effect size differs according to sample size are shown in Table 6. e-ISSN: 2536-4758 http://www.efdergi.hacettepe.edu.tr/ The Effect of Flipped Learning Approach on Academic Achievement: A Meta-Analysis Study 11 Table 6: Findings regarding the effect size according to sample size Variable QB p Sample Size 1-30 participants 31-60 participants 61 or more participants 9.145 .01 n ES Effect Size at 95% Confidence Interval ESMin SE ESMax 11 39 30 1.056 .668 .358 .578 .474 .155 .244 .099 .104 1.535 .862 .561 It is found that critical value obtained from the chi-square table at 95% significant level with two degrees of freedom is 5.991. The homogeneity value among the groups according to sample size is (QB) 9.145. Since Q value is larger than the critical value obtained from the chisquare table at 95% significant level with two degrees of freedom, there is a statistically significant difference among the groups according to sample size. Accordingly, it proves that the effect of the flipped learning approach on academic achievement is higher in small groups than in large groups. 3.5. Findings Regarding the Research Question: “Does the effect of the flipped learning approach on academic achievement differ according to the practitioner?” Findings regarding whether the effect size differs according to the practitioner are shown in Table 7. Table 7: Findings regarding the effect size according to the practitioner Variable QB p Practitioner Researcher Teacher Researcher/Teacher Unspecified 1.686 .640 n ES Effect Size at 95% Confidence Interval ESMin SE ESMax 28 34 8 10 .568 .565 .793 .404 .317 .358 .352 .017 .129 .106 .225 .198 .820 .772 1.233 .792 It is found that critical value obtained from the chi-square table at 95% significant level with three degrees of freedom is 7.815. The homogeneity value among the groups according to the practitioner is (QB) 1.686. Since Q value is smaller than the critical value obtained from the chi-square table at 95% significant level with three degrees of freedom, there is not a statistically significant difference among the groups according to the practitioner. 3.6. Findings Regarding the Research Question: “Does the effect of the flipped learning approach on academic achievement differ according to being national/international?” Findings regarding whether the national/international are shown in Table 8. effect size differs according to being Table 8: Findings regarding the effect size according to being national/international Variable National International QB p 13.265 .000 n ES Effect Size at 95% Confidence Interval ESMin SE ESMax 26 54 .946 .408 .700 .256 .126 .078 1.192 .560 It is found that critical value obtained from the chi-square table at 95% significant level with one degree of freedom is 3.841. The homogeneity value among the groups according to being national/international is (QB) 13.265. Since Q value is larger than the critical value e-ISSN: 2536-4758 http://www.efdergi.hacettepe.edu.tr/ İbrahim Karagöl, Emrullah Esen 12 obtained from the chi-square table at 95% significant level with one degree of freedom, there is a statistically significant difference among the groups according to being national/international. Accordingly, it proves that the effect of the flipped learning approach on academic achievement is higher in national studies than in international studies. In order to unveil the reasons for this difference, it is necessary to determine from which perspectives national studies differ from international studies. In this respect, studies included in the meta-analysis were examined in terms of the implementation period and sample size according to being national or international. First, descriptive statistics of the implementation period of the flipped learning approach according to whether studies are national or international are given in Table 8a. Table 8a: Descriptive national/international Implementation Period National International Total statistics of 1-4 weeks f % 10 38.4 4 7.4 14 the implementation 5-8 weeks f % 13 50.0 25 46.3 38 period according to being 9 or more weeks f % 3 11.5 17 31.5 20 Table 8a shows that the number of the studies of which implementation period corresponds to the time interval of 5-8 weeks is high in number (38). It draws attention that national studies (38.4%) corresponding to the time interval of 1-4 weeks are more than international ones (7.4%), and the studies corresponding to the time interval of 9 or more weeks are seen to be high in number in international studies. The frequency distribution of 8 studies (14.8%) that did not report the implementation period was excluded from Table 8a. As another factor that may influence the effect size, the sample sizes of the experimental groups are determined. Descriptive statistics of the sample size of the experimental group according to whether studies are national or international are given in Table 8b. Table 8b: Descriptive statistics of the sample size according to being national/international Sample Size National International Total 1-30 participants F % 9 34.6 2 3.7 11 31-60 participants f % 12 46.1 27 50.0 39 61 or more participants f % 5 19.2 25 46.3 30 As shown in the table 8b, the number of the studies whose sample size is between 31-60, is higher in number (39) compared to the others. It is noteworthy that national studies with a sample size of 1-30 (34.6%) are more than international ones (3.7%). Moreover, international studies with a sample size of 61 or more participants (46.3%) are seen to be high in number than national ones (19.2%) in the groups. 4. RESULTS and DISCUSSION This study aims to examine the effect of the flipped learning approach on academic achievement through meta-analysis method. Within the scope of the first research question, which examines the effect of flipped learning on students’ academic, 80 datasets were included in the meta-analysis. The homogeneity value of the studies (Q = 434.834) included in the study exceeds the critical value of the chi-square distribution with 79 degrees of freedom (for df=79, X2 (0.95)=100.749). Therefore, the random effects model was preferred to calculate the effect size of the studies. According to the results of the analysis through random effects model, general effect sizes of the studies were found as .566, which has moderate effect according to the classification by Cohen et al. (2007). Therefore, it is concluded that the effect of the flipped learning approach on academic achievement is more effective than traditional learning e-ISSN: 2536-4758 http://www.efdergi.hacettepe.edu.tr/ The Effect of Flipped Learning Approach on Academic Achievement: A Meta-Analysis Study 13 approach. Similarly, in a meta-analysis study by Hew & Lo (2018), which examines the effect of flipped learning in health profession on student learning, it is found a significant effect in favour of flipped learning for health professionals’ education (SMD = 0.33, 95% CI 0.21–0.46, p < .001). There can be various variables that can affect academic achievement in flipped context. For example, it is reported that, in a flipped learning environment, students take more responsibility (Educause, 2012), learn at their own pace and style (Fulton, 2012), participate in interactive and problem-solving activities as active learners (Sharpe, 2016), and show higher level of motivation (Tarazi, 2016. Additionally, such an environment causes an increase in students’ awareness of metacognition (Yıldız & Kıyıcı, 2016) and their conceptual understanding (Renfro, 2014) allowing group discussion and peer instruction (Fulton, 2012). Furthermore, teachers use “just-in-time teaching” to adjust their instruction according to questions and web-based quizzes done by students before class (Berett, 2012). These benefits and characteristics of flipped learning approach might be the reason that leads a better academic achievement. The second research question is “Does the effect of the flipped learning approach on academic achievement differ according to educational level?” When the effect sizes of educational level of students are compared, it is seen that the highest effect size value belongs to elementary school level (ES=.653) and the lowest effect size value belongs to high school level (ES=.448). Besides, homogeneity value among the groups (QB=1.100) is smaller than the critical value obtained from the chi-square table at 95% significant level with two degrees of freedom (for df=2, X2 (0.95)=5.991). Therefore, it is concluded that there is not a statistically significant difference among the groups according to educational level. The third research question is: “Does the effect of the flipped learning approach on academic achievement differ according to the implementation period?” When the effect sizes of implementation periods are compared, it is seen that effect size is .692 for 1-4 weeks, .582 for 58 weeks, and .410 for 9 or more weeks. The homogeneity value among the groups (QB= 2.206) is smaller than the critical value obtained from the chi-square table at 95% significant level with three degrees of freedom (for df=3, X2 (0.95)=7.815). Therefore, it is concluded that there is not a statistically significant difference among the groups according to the implementation period. Even though there is not a significant difference among the groups, after four weeks, as the implementation period of the flipped learning approach lengthens, the effect size reduces. The fourth research question is: “Does the effect of the flipped learning approach on academic achievement differ according to sample size?” When the effect sizes of sample sizes are compared, the highest effect size value belongs to groups with a sample size of 1-30 (ES=1.056) and the lowest effect size value belongs to groups with a sample size of 61 or more participants (ES=.358). Since the homogeneity value among the groups (QB=9.145) is larger than the critical value obtained from the chi-square table at 95% significant level with two degrees of freedom (for df=2, X2 (0.95)=5.991), there is a statistically significant difference among the groups according to sample size. Therefore, it can be concluded that the effect of flipped learning on academic achievement is higher in small groups than in large groups. That is, as the groups get larger, the effect size reduces. Based on the findings, it can be inferred that implementations are more successful in small groups than in large groups because small groups may facilitate the implementation of flipped learning approach and help teachers have control over classroom easily. In this regard, it is reported by other researchers that teaching is more effective and students are more successful in small classes (Burgaz, 2002; McGiverin, Gilman & Tillitski, 1989; Nye, Hedges & Konstantopoulos, 2000; Resnick & Zurawsky, 2003). Moreover, teachers indicated that smaller classes have many advantages than larger classes, and their experiences are better in smaller classes especially in terms of individualisation (Shapson, e-ISSN: 2536-4758 http://www.efdergi.hacettepe.edu.tr/ İbrahim Karagöl, Emrullah Esen 14 Wright, Eason & Fitzgerald, 1980, p. 149). In a meta-analysis study by McGiverin, Gilman & Tillitski (1989), it is found that primary school students who studied in small classes for three years are more successful than their peers who studied in crowded classes. Finn, Pannozzo & Achilles (2003) also pointed out that small classes increase students’ academic performance and help enhance students’ engagement in the classroom. The fifth research question is: “Does the effect of the flipped learning approach on academic achievement differ according to the practitioner?” In terms of researchers, teachers and researchers/teachers as practitioners, the effect size of the studies is found respectively .568, .565 and .793. The homogeneity value among the groups (QB=1.686) is smaller than the critical value obtained from the chi-square table at 95% significant level with three degrees of freedom (for df=3, X2 (0.95)=7.815). Therefore, it is concluded that there is not a statistically significant difference among the groups according to the practitioner. The sixth research question is: “Does the effect of the flipped learning approach on academic achievement differ according to being national/international?” When the effect sizes of studies are compared in terms of being national/international, it is seen that effect size is .946 for national studies and .408 for international studies. The homogeneity value between the groups (QB=13.265) is larger than the critical value obtained from the chi-square table at 95% significant level with one degree of freedom (for df=1, X2 (0.95)=3.841). There is a statistically significant difference between the groups according to being national/international. Therefore, it inferred that the effect of the flipped classroom on academic achievement is higher in national studies than in international studies. In order to unveil the reasons for this difference, national and international studies were examined in terms of the implementation period and sample size. It is seen that national studies (38.4%) corresponding to the time interval of 1-4 weeks are more than international ones (7.4%), and the studies corresponding to the time interval of 9 or more weeks are seen to be high in number in international studies. Besides, national studies with a sample size of 1-30 (34.6%) are more than international ones (3.7%), and international studies are high in number than national ones in the groups with a sample size of 61 or more participants. As stated above, the effect sizes are high when the implementation is carried out with small groups (1-30) and the implementation period lasts between 1-4 weeks. Therefore, the higher effect size in the national studies might stem from the fact that these studies were carried out with small groups and their implementation period lasted between 1-4 weeks. According to the results of the study, flipped learning approach is more effective at the elementary school level as educational level; between 1-4 weeks as implementation period and between 1-30 participants as the sample size for increasing the academic achievement of students. Besides, flipped learning approach is less effective at the high school level as educational level; over 9 weeks as implementation period and over 31 participants as the sample size in terms of the academic achievement of students. Considering the results obtained in the study, the following implications can be made:  Flipped learning approach is found to be more effective between 1-30 participants as sample size. Thus, the flipped learning approach should be used more in classes with 30 students or less in order to enhance students’ performance.  It is concluded that after four weeks, as the implementation period of the flipped learning approach lengthens, the effect size reduces. It can be recommended that the effect of the implementation period should be researched in a detailed way for further studies.  This study examines the effect of flipped learning on only academic achievement. The effect of flipped learning on factors such as attitude, self-efficacy, and motivation can be analysed through meta-analysis for further studies. e-ISSN: 2536-4758 http://www.efdergi.hacettepe.edu.tr/ The Effect of Flipped Learning Approach on Academic Achievement: A Meta-Analysis Study  15 When the effect size of the flipped learning approach on students’ academic achievement is examined, it is found that there is a significant difference according to being national/international. The effect size is higher in national studies than international ones. The reasons for this result can be more deeply investigated. 5. REFERENCES (* marked references are the studies that are included in the meta-analysis) Abeysekera, L., & Dawson, P. (2014) Motivation and cognitive load in the flipped classroom: definition, rationale and a call for research. Higher Education Research & Development, 34(1), 1-14. *Aidinopoulou, V., & Sampson, D. G. (2017). An action research study from implementing the flipped classroom model in primary school history teaching and learning. Journal of Educational Technology & Society, 20(1), 237. *Akgün, M., & Atıcı, B. (2017). Ters-düz sınıfların öğrencilerin akademik başarısı ve görüşlerine etkisi. Kastamonu Education Journal, 25(1). Akobeng, A. K. (2005). Principles of evidence based medicine. Archives of disease in childhood, 90(8), 837-840. *AlJaser, A. M. (2017). Effectiveness of using flipped classroom strategy in academic achievement and self-efficacy among education students of princess Nourah bint Abdulrahman University. English Language Teaching, 10(4), 67-77. Arnold-Gaza, S. (2014). The flipped classroom: A survey of the research. Communications In Information Literacy, 8(1), 8-22. *Aydın, B. (2016). Ters yüz sınıf modelinin akademik başarı, ödev/görev stres düzeyi ve öğrenme transferi üzerindeki etkisi. Unpublished master's dissertation. Süleyman Demirel University, Isparta. *Aydın, G. (2016). Ters yüz sınıf modelinin üniversite öğrencilerinin programlamaya yönelik tutum, öz-yeterlik algısı ve başarılarına etkisinin incelenmesi. Unpublished master's dissertation. Dokuz Eylül University, İzmir. Baker, J. W. (2000). The “Classroom Flip”: Using web course management tools to become the guide by the side. In J. A. Chambers (Ed.), Selected papers from the 11th International Conference on College Teaching and Learning, 9-17. Jacksonville, FL: Florida Community College at Jacksonville. Bergman, J., & Sams, A. (2012). Flip your classroom: Reach every student in every class every day. Washington: ISTE. Berrett, D. (2012). How 'Flipping' the classroom can improve the traditional lecture. Chronicle of Higher Education, 58(25), 16-18. *Bhagat, K. K., Chang, C. N., & Chang, C. Y. (2016). The Impact of the flipped classroom on mathematics concept learning in high school. Educational Technology & Society, 19(3), 134-142. *Bishop, J. L. (2013). A controlled study of the flipped classroom with numerical methods for engineers. Unpublished doctoral dissertation. Utah State University. Bishop, J. L., & Vergleher, M. A. (2013). The flipped classroom: A survey of the research. In ASEE National Conference Proceedings, Atlanta, GA, 30 (9), 1-18. Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. West Sussex, UK: John Wiley & Sons, Ltd. Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2013). Meta-analize giriş (Çev. Serkan Dinçer). Ankara: Anı Yayımcılık. *Boyraz, S., & Ocak, G. (2017). The implementation of flipped education into Turkish EFL teaching context. Journal of Language and Linguistic Studies, 13(2), 426-439. Brame, C. J. (2013). Flipping the classroom. [Available online at: https://cft.vanderbilt.edu/wpcontent/uploads/sites/59/Flipping-the-classroom.pdf], Retrieved on June 29, 2018. *Brooks, A. (2014). Information literacy and the flipped classroom: Examining the impact of a one-shot flipped class on student learning and perceptions. Communications in Information Literacy, 8(2), 4. Burgaz, B. (2002). Kalabalık sınıf nitelikli öğretmen. Bilim ve Teknik Dergisi, 420. e-ISSN: 2536-4758 http://www.efdergi.hacettepe.edu.tr/ 16 İbrahim Karagöl, Emrullah Esen *Cashin, M. (2016). The effect of flipped classrooms on elementary students' reading scores. Unpublished doctoral dissertation. Northcentral University. *Chao, C. Y., Chen, Y. T., & Chuang, K. Y. (2015). Exploring students' learning attitude and achievement in flipped learning supported computer aided design curriculum: A study in high school engineering education. Computer Applications in Engineering Education, 23(4), 514-526. *Chiu, W.L. (2016). Effects of flipped and traditional learning on college students' English learning outcomes and motivation. Unpublished master's dissertation. National Sun Yat-sen University. *Clark, K. R. (2013). Examining the effects of the flipped model of instruction on student engagement and performance in the secondary mathematics classroom: An action research study. Unpublished doctoral dissertation. Capella University. Cohen, L., Manion, L., & Morrison, K. (2007). Research methods in education. New York: Routledge. *Crawford, T. H. (2017). Flipped learning influence on active learning and assessments in the postsecondary hospitality classroom: An action research study. Unpublished doctoral dissertation. Capella University. *Çakır, E. (2017). Ters yüz sınıf uygulamalarının fen bilimleri 7. sınıf öğrencilerinin akademik başarı, zihinsel risk alma ve bilgisayarca düşünme becerileri üzerine etkisi. Unpublished master's dissertation. Ondokuz Mayıs University, Samsun. Davies, P. (2000). The relevance of systematic reviews to educational policy and practice. Oxford Review of Education, 26(3-4), 365-378. *Davies, R. S., Dean, D. L., & Ball, N. (2013). Flipping the classroom and instructional technology integration in a college-level information systems spreadsheet course. Educational Technology Research and Development, 61(4), 563-580. Dempfle, A. (2006). Evaluation of methods for meta-analysis of genetic linkage studies for complex diseases and application to genome scans for asthma and adult height. Unpublished doctoral dissertation. Philipps-University Marburg, Marburg. *Dixon, K. L. (2017). The effect of the flipped classroom on urban high school students’ motivation and academic achievement in a high school science. Unpublished doctoral dissertation. Liberty University, Lynchburg, VA. *Dove, A., & Dove, E. (2015). Examining the influence of a flipped mathematics course on preservice elementary teachers' mathematics anxiety and achievement. Electronic Journal of Mathematics & Technology, 9(2), 167179. *Duffy, C. M. (2016). The impact of flipped learning on student achievement in an eighth grade earth science classroom. Unpublished doctoral dissertation. Wilkes University. Educause (2012). Things you should know about flipped classrooms. [Available online https://library.educause.edu/~/media/files/library/2012/2/eli7081-pdf.], Retrieved on May 29, 2018. at: Ellis, P. (2010). The essentials of effect sizes. New York: Cambridge University Press. Enfield, J. (2013). Looking at the impact of the flipped classroom model of instruction on undergraduate multimedia students at CSUN. TechTrends, 57(6), 14-27. *Faretta, R. S. (2016). A causal-comparative inquiry into the significance of implementing a flipped classroom strategy in nursing education. Unpublished doctoral dissertation. Liberty University. *Files, D. D. (2016). Instructional approach and mathematics achievement: an investigation of traditional, online, and flipped classrooms in college algebra. Unpublished doctoral dissertation. Florida Institute of Technology. Finn, J. D., Pannozzo, G. M., & Achilles, C. M. (2003). The why's of class size: Student behaviour in small classes. Review of Educational Research, 73, 321-368. *Fraga, L. M., & Harmon, J. (2014). The flipped classroom model of learning in higher education: An investigation of preservice teachers’ perspectives and achievement. Journal of Digital Learning in Teacher Education, 31(1), 18-27. Fulton, K. P. (2012). 10 Reasons to Flip. New Styles of Instruction, 94(2), 20-24. Harris, B., Harris, J., Reed. L., & Zelihic, M. (2016). Flipped classroom: Another tool for your pedagogy tool box. Developments in Business Simulation and Experiential Learning, 1(43), 325-333. Hew, K. F., & Lo, C. K. (2018). Flipped classroom improves student learning in health professions education: A meta-analysis. BMC Medical Education, 18(1), 38. e-ISSN: 2536-4758 http://www.efdergi.hacettepe.edu.tr/ The Effect of Flipped Learning Approach on Academic Achievement: A Meta-Analysis Study 17 *Howell, D. (2013). Effects of an Inverted Instructional Delivery Model on Achievement of Ninth-Grade Physical Science Honors Students. Unpublished doctoral dissertation. Gardner-Webb University. *Johnson, W. L. (2012). Effect of the flipped classroom model on a secondary computer applications course: Student and teacher perceptions, questions and student achievement. Unpublished doctoral dissertation. University of Louisville. Louisville, Kentucky. *Kurt, G. (2017). Implementing the flipped classroom in teacher education: Evidence from Turkey. Educational Technology & Society, 20(1), 211–221. Lage, M. J., Platt, G. J., & Treglia, M. (2000). Inverting the classroom: A gateway to creating an inclusive learning environment. The Journal of Economic Education, 31(1), 30-43. *Lee, B. H., & Lee, H. C. (2016). The effects of science lesson with the application of flipped learning on science academic achievement and scientific attitude. Journal of Korean Elementary Science Education, 35(1), 78-88. McGiverin, J., Gilman, C., & Tillitski, A. (1989). Meta-analysis of the relation between class size and achievement. Elementary School Journal 90(1), 47–56. Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis. Thousand Oaks, CA: Sage Publications. *Montgomery, J. (2015). The effects of flipped learning on middle school students' achievement with common core mathematics. Unpublished master thesis. California State University, San Marcos. Nye, B., Hedges, L. V., & Konstantopoulos, S. (2000). The effects of small classes on academic achievement: The results of the Tennessee class size experiment. American Educational Research Journal, 37(1). 123-151. *Olakanmi, E. E. (2017). The effects of a flipped classroom model of ınstruction on students’ performance and attitudes towards chemistry. Journal of Science Education and Technology, 26(1), 127-137. *Overmyer, G. R. (2014). The flipped classroom model for college algebra: Effects on student achievement. Unpublished doctoral dissertation. Colorado State University. *Özpinar, İ., Yenmez, A. A., & Gökçe, S. (2016). An Application of Flipped Classroom Method in the Instructional Technologies and Material Development Course. Journal of Education and Training Studies, 4(12), 213-226. *Peterson, D. J. (2016). The flipped classroom improves student achievement and course satisfaction in a statistics course: A quasi-experimental study. Teaching of Psychology, 43(1), 10-15. Petitti, D. B. (2000). Meta-analysis, decision analysis and cost effectiveness analysis: Methods for quantitative synthesis in medicine. NY: Oxford University Press. Pierce, R., & Fox, J. (2012). Vodcasts and active-learning exercises in a “Flipped classroom” model of a renal pharmacotherapy module. American journal of pharmaceutical education, 76(10), 196. Pigott, T. (2012). Advances in meta-analysis. London: Springer Science & Business Media. *Prefume, Y. E. (2015). Exploring a flipped classroom approach in a Japanese language classroom: a mixed methods study. Unpublished doctoral dissertation. Baylor University . *Renfro, A. (2014). Assessing the effects of a flipped learning approach on student achievement, mathematical thinking, attitudes, and teacher perceptions in an undergraduate calculus class using a participatory action research approach. Unpublished doctoral thesis, Robert Morris University. Resnick, L. B., & Zurawsky, C. (2003). Class size: Counting students can count. American Educational Research Association, 1(2), 1-4. Ried, K. (2006). Interpreting and understanding meta-analysis graphs. Australian Family Physician, 35(8), 635-638. *Ripley, D. (2015). An Examination of flipped ınstructional method on sixth graders’ mathematics learning: utilizing propensity score matching. Unpublished doctoral dissertation. University of Nevada. *Sağlam, D. (2016). Ters-yüz sınıf modelinin İngilizce dersinde öğrencilerin akademik başarılarına ve tutumlarına etkisi. Unpublished master's dissertation. Bülent Ecevit University, Zonguldak. *Salimi, A., & Yousefzadeh, M. (2015). The effect of flipped learning (revised learning) on Iranian students’ learning outcomes. Advances in Language and Literary Studies, 6(5), 209-213. Sankoff, P. (2014). Taking the instructions of law outside of the lecture hall: How the Flipped classroom can make learning more productive and enjoyable (for professors and students). Alberta Law Review, 51(4), 891-906. Sarıer, Y. (2016). Türkiye’de öğrencilerin akademik başarısını etkileyen faktörler: Bir meta-analiz çalışması. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 31(3), 1-19. e-ISSN: 2536-4758 http://www.efdergi.hacettepe.edu.tr/ 18 İbrahim Karagöl, Emrullah Esen *Saunders, J. M. (2014). The flipped classroom: its effect on student academic achievement and critical thinking skills in high school mathematics. Unpublished doctoral thesis. Liberty University, Virginia. *Schwanki, E. R. (2013). Blended learning: achievement and perception flipped classroom: effects on achievement and student perception. Unpublished master's dissertation. Southwest Minnesota State University. *Sezer, B. (2017). The effectiveness of a technology-enhanced flipped science classroom. Journal of Educational Computing Research, 55(4), 471-494. Shapson, S. M., Wright, E. N., Eason, G., & Fitzgerald, J. (1980). An experimental study of effects of class size. American Educational Research Journal 17, 141–152. *Sharpe, E. H. (2016). An investigation of the flipped classroom in algebra two with trigonometry classes. Unpublished doctoral thesis. Regent University, London. *Sickle, V. J. (2016). Discrepancies between student perception and achievement of learning outcomes in a flipped classroom. Journal of the Scholarship of Teaching and Learning, 16(2), 29-38. *Smith, J. P. (2015). The efficacy of a flipped learning classroom. Unpublished doctoral dissertation. McKendree University. Staker, H., & Horn, M. B. (2012). Classifying K-12 blended learning. San Francisco, CA: Innosight Institute. *Sun, J. C. Y., & Wu, Y. T. (2016). Analysis of learning achievement and teacher–student interactions in flipped and conventional classrooms. The International Review of Research in Open and Distributed Learning, 17(1), 79-99 *Tarazi, N. (2016). The influence of the inverted classroom on student achievement and motivation for learning in secondary mathematics in the United Arab Emirates: A quasi-experimental study. Unpublished doctoral dissertation. Northcentral University. *Thai, N. T. T., De Wever, B., & Valcke, M. (2017). The impact of a flipped classroom design on learning performance in higher education: Looking for the best “blend” of lectures and guiding questions with feedback. Computers & Education, 107, 113-126. *Topalak, . (2016). evrilmiş öğrenme modelinin başlangı seviyesi piyano öğretimine etkisi. Unpublished doctoral dissertation. Doktora Tezi. İnönü Üniversitesi, Malatya. Tune, J. D., Sturek, M., Basile, D. P. (2013). Flipped classroom model improves graduate student performance in cardiovascular, respiratory, and renal physiology. Advances in Physiology Education, 37, 316–320. *Turan, Z. (2015). Ters yüz sınıf yöntemının değerlendırılmesı ve akademık başarı, bılışsel yük ve motıvasyona etkısının ıncelenmesı. Unpublished doctoral dissertation. Atatürk University, Erzurum. *Umutlu, D. (2016). Effects of different video modalities on writing achievement in flipped english classes. Unpublished master's dissertation. Boğaziçi University, İstanbul. Üstün, U. ve Eryılmaz, A. (2014). Etkili araştırma sentezleri yapabilmek için bir araştırma yöntemi: Meta-analiz. Eğitim ve Bilim, 174(39), 1-32. *Vang, Y. V. (2017). The impact of the flipped classroom on high school mathematics students' academic performance and self-efficacy. Unpublished doctoral dissertation. California State University. *Webb, M., & Doman, E. (2016). Does the flipped classroom lead to increased gains on learning outcomes in ESL/EFL contexts?. CATESOL Journal, 28(1), 39-67. *Wiginton, B. (2013). Flipped instruction: An investigation into the effect of learning environment on student selfefficacy, learning style, and academic achievement in an algebra I classroom. Unpublished doctoral dissertation. University of Alabama Libraries. *Winter, J. B. (2013). The effect of the flipped classroom model on achievement in an introductory college physics course. Unpublished doctoral dissertation. Mississippi State University. Wolf, F. M. (1986). Meta-analysis: Quantitative methods for research synthesis. London: Sage. *Yavuz, M. (2016). rtaöğret m düzeyinde ters yüz sınıf uygulamalarının akademik başarı üzerine etkisi ve öğrenci deneyimlerinin incelenmesi. Unpublished master’s dissertation. Atatürk University, Erzurum. *Yıldız, D. G., & Kıyıcı, G. (2016). Ters-yüz edilmiş sınıf modelinin öğretmen adaylarının erişileri ve görüşleri açısından incelenmesi. Sakarya University Journal of Education, 6(3), 186-200. *Zhonggen, Y., & Wang, G. (2016). Academic achievements and satisfaction of the clicker-aided flipped business English writing class. Journal of Educational Technology & Society, 19(2), 298. e-ISSN: 2536-4758 http://www.efdergi.hacettepe.edu.tr/ The Effect of Flipped Learning Approach on Academic Achievement: A Meta-Analysis Study 19 UZUN ÖZET Günümüzde yaşanan bilimsel ve teknolojik gelişmelerle birlikte, eğitim-öğretim ortamları farklılaşmaya ve değişmeye başlamıştır. Özellikle, bilgisayar ve internet kullanımının yaygınlaşması çoklu ortam teknolojilerinin eğitim ortamlarına dahil edilme sürecine hız kazandırmıştır. Teknolojinin yaygınlaşması ile birlikte değişen ihtiyaçlar ve artan olanaklar eğitimde uzaktan eğitim, bilgisayar destekli öğretim, harmanlanmış öğretim ve internet tabanlı öğretim gibi yenilikçi yaklaşımların hayata geçirilmesine zemin hazırlamıştır. Bu doğrultuda ortaya çıkan yeni yaklaşımlardan biri de ters-yüz edilmiş öğrenme yaklaşımıdır. Geleneksel yöntemlerin aksine, ters-yüz edilmiş öğrenme yaklaşımının temel mantığı doğrudan anlatım etkinliklerini dersin dışına taşıyarak, ders süresince, işbirlikçi aktivitelere ve pratik uygulamalara daha fazla zaman ayırmaktır (Arnold-Garza, 2014; Bergman & Sams, 2012; Bishop & Vergler, 2013; Enfield, 2013). Ders içerikleri, öğrenciler tarafından kısa videolar ya da ses kayıtları aracılığıyla takip edilirken, ders saati öğrencilerin öğrendiklerini uygulayabilecekleri egzersizlere, birbirleriyle iş birliği yapabilecekleri projelere ve birbirleriyle etkileşim kurabilecekleri çeşitli etkinliklere ayrılır (Educause, 2012). Akademik başarı açısından bakıldığında ters-yüz edilmiş ve geleneksel sınıflar pek çok araştırmanın konusu olmuştur. Son test puanları temel alındığında, yürütülen bazı çalışmalar ters-yüz edilmiş sınıflar lehine anlamlı bir başarı farkı olduğunu ortaya koyarken (Aljeser, 2007; Aydın, 2016; Sun & Wu, 2016; Turan, 2015, Salimi & Yousefzadeh, 2015;), bazı çalışmalar ise son-test puanları arasında anlamlı bir fark olmadığını (Dixon, 2017; Fraja, 2017; Sharpe, 2016; Saunders, 2014; Yavuz, 2016) ortaya koymaktadır. Araştırma sonuçları ters-yüz edilmiş öğrenme yaklaşımının akademik başarı üzerindeki etkisine yönelik farklı bulgular ortaya koymaktadır. Eğitim bilimleri alanının karmaşık bir yapıya sahip olması; aynı zamanda çalışmaların iç geçerliğini etkileyen ve ortadan kaldırılması mümkün olmayan çok sayıda tehdidin bulunması (Üstün & Eryılmaz, 2014) bu durumun sebepleri arasında gösterilebilir. Diğer taraftan, Davies (2000) tek başına bir deneyin; zaman, örneklem ve bağlam gibi duruma özgü sınırlılıkları olduğunu ifade etmektedir. Bu bağlamda aynı konu üzerinde yapılmış farklı araştırma sonuçlarının bulgularını bir araya getirerek, örneklemi genişletmek ve güvenilir sonuçlar elde etmek için kullanılan meta-analiz araştırmalarının (Dempfle, 2006; Petitti, 2000) önemli bir yere sahip olduğu belirtilmektedir. Bu doğrultuda ters-yüz edilmiş öğrenme yaklaşımının akademik başarıya olan etkisini ortaya çıkarmak amacıyla ülkemizde ve dünyada yapılmış araştırmalara yönelik bir meta-analiz araştırmasına ihtiyaç olduğu düşünülmektedir. Çalışma kapsamında ters-yüz edilmiş öğrenme yaklaşımının akademik başarıya etkisinin meta-analiz yöntemiyle incelemesi amaçlanmıştır. Araştırmada ters-yüz öğrenme yaklaşımının akademik başarıya olan etkisini incelemek için metaanaliz yöntemi kullanılmıştır. Araştırmaya dahil edilecek çalışmaları, ters-yüz edilmiş öğretimle ilgili araştırma problemlerine ve gerekli istatistiksel verilere sahip olan bilimsel dergilerde yayımlanmış makaleler ile yüksek lisans ve doktora tezleri oluşturmaktadır. Araştırma kapsamında “ters-yüz edilmiş öğretim”, “ters-yüz edilmiş öğrenme”, “akademik başarı” gibi anahtar kelimeler kullanılarak EBSCOhost, ProQuest, JSTOR, Google Akademik, TÜBİTAK ULAKBİM Sosyal Bilimler Veri Tabanı, YÖK Tez ve ERIC arama motorları aracılığıyla ilgili tez ve makaleler elde edilmiştir. Ulaşılan çalışmaların özet kısımları okunmuş ve eş kopyalar elendikten sonra incelemeye 365 çalışma alınmıştır. Kısıtlamalı olan dört teze ise ulaşılamamıştır. Araştırma problemine uygun olamayan (n=258) ve dahil edilme ölçütlerini karşılamayan (n=51) makale ve tezler elenmiştir. Eğer aynı çalışma hem tez hem de makale olarak yayımlanmışsa, daha detaylı bilgi içereceği için tezler araştırma kapsamına alınmış, makaleleri alınmamıştır (n=1). Böylelikle geriye 55 çalışma kalmıştır. Bazı çalışmalarda ters-yüz edilmiş öğrenme yaklaşımının akademik başarıya etkisi her bir ders için ayrı ayrı hesaplandığı için veri seti 80 olarak belirlenmiştir. Meta-analiz kodlama sürecinde çalışmaların meta-analize dahil edilme ölçütlerine uygun olup olmadığının belirlenmesi ve çalışmalar arasında karşılaştırma yapılabilmesi için çalışmanın amacına uygun olarak bir kodlama formu oluşturulmuştur. Kodlama formunda çalışmanın adı, çalışmanın yazarı, çalışmanın yapıldığı yer, çalışmanın uygulama süresi, çalışmayı kimin yürüttüğü, örneklem düzeyi, çalışmadaki toplam örneklem sayısı, çalışmadaki istatistiksel veriler gibi çalışmaya ait birtakım bilgiler istenilmiştir. Araştırmada kodlamalar araştırmacılar tarafından yapılmıştır. Bununla birlikte, rasgele seçilen 11 çalışmanın kodlaması eğitim bilimleri alanında doktora eğitimine devam eden başka bir araştırma görevlisi tarafından yapılmıştır. Kodlama güvenirliğinin hesaplanabilmesi için bu 11 çalışmanın kodlanan bulguları karşılaştırılmış, Miles & Huberman’ın (1994) güvenirlik formülü kullanılarak görüşler e-ISSN: 2536-4758 http://www.efdergi.hacettepe.edu.tr/ 20 İbrahim Karagöl, Emrullah Esen arasındaki güvenirlik katsayısı hesaplanmıştır. Hesaplamalar sonucunda birinci araştırmacı ile araştırma görevlisi arasındaki güvenirlik katsayısı %93, ikinci araştırmacı ile araştırma görevlisi arasındaki güvenirlik katsayısı %92, araştırmacıların kendi arasındaki güvenirlik katsayısı %99 bulunmuştur. Güvenirlik katsayısının %70’in üzerinde olması araştırma için güvenilir kabul edilmektedir (Miles & Huberman, 1994). Dolayısıyla kodlamaların güvenilir olduğu söylenebilir. Örtüşmeyen kodlamalar, kodlayıcılar tarafından tekrar gözden geçirilmiş ve ortak bir kararla düzeltilmiştir. Yapılan meta-analiz araştırması sonucunda ters-yüz edilmiş öğrenme yaklaşımının akademik başarı üzerinde pozitif yönde etkisi olduğu görülmüştür. Ters-yüz edilmiş öğrenme yaklaşımının akademik başarıya ilişkin genel etki büyüklüğü değeri rastgele etkiler modeli kullanılarak, %95 güven aralığının alt sınırı .428, üst sınırı .705 ve etki büyüklüğünün ortalama değeri .566 olarak hesaplamıştır. Cohen ve diğerlerine (2007) göre bu değer orta düzeyde bir etkiye sahiptir. Araştırmanın yayım yanlılığını ortaya çıkarmak amacıyla “Classic Fail-Safe N” analizinden ve Huni Grafiğinden (Funnel Plot) yararlanılmış, meta-analiz araştırmasında yayım yanlılığının olmadığı belirlenmiştir. Ters-yüz edilmiş öğrenme yaklaşımının akademik başarı üzerindeki etkisinin uygulama sürelerinin uzunluğuna ya da kısalığına göre değişmediği; ters-yüz edilmiş öğrenme yaklaşımının akademik başarı üzerindeki etkisinin küçük gruplarda daha yüksek olduğu; ters-yüz edilmiş öğrenme yaklaşımının uygulayan kişinin öğrenci başarısı üzerinde anlamlı bir etkisinin olmadığı ve çalışmaların ulusal/uluslararası olma durumuna göre gruplar arasında anlamlı bir farklılık olduğu sonuçlarına ulaşılmıştır. Araştırma sonuçlarından elde edilen bulgulara dayanarak, ters-yüz edilmiş öğrenme yaklaşımının sınıf mevcudu olarak 30 ve daha az sınıf büyüklüklerinde kullanılması önerilebilir. Ters-yüz edilmiş öğrenme yaklaşımının öğrencilerin akademik başarısına ilişkin etki büyüklükleri incelendiğinde çalışmaların ulusal/uluslararası olma durumuna göre anlamlı farklılık tespit edilmiştir. Etki büyüklüğü ulusal çalışmalarda daha yüksek çıkmıştır. İleriki araştırmalar için bu sonuçların nedenleri derinlemesine araştırılabilir. e-ISSN: 2536-4758 http://www.efdergi.hacettepe.edu.tr/