Evaluation of a French immersion program: The Ottawa study through Grade five

Evaluation of a French immersion program: The Ottawa Study through grade five HENRI C. BARIK AND MERRILL SWAIN The Ontario Institute for Studies in Education ABSTRACT The results of the evaluation of the French immersion program in grades three to five of the Ottawa public school system indicate that pupils in the program are in general on a par with or ahead of their peers in the regular English program in most academic areas considered (English, mathematics, work-study skills, science) and are performing satisfactorily in French. Thesefindingsare consistent with those of other evaluations. In the Canadian educational context, French immersion refers to a program in which English-speaking pupils attending English schools are taught solely or in part via French. The concept has gained wide acceptance since it was first introduced to the private school sector in 1962 by the establishment of the Toronto French School, and adopted by the public school system of St. Lambert, Quebec, in 1965 (Lambert & Tucker, 1972; see Swain, 1976, for research-oriented bibliography; Bilingual Education Project, 1976, for list of immersion programs). The present paper reports on the 1975-76 evaluation of the French immersion program in the Ottawa public school system, which at present receives close to one-third of the children entering the system in kindergarten. DESCRIPTION OF OTTAWA PROGRAM The Ottawa public school French immersion program was initiated in 1970 at the kindergarten (K) level in several schools under the jurisdictions of the Ottawa Board of Education and the Carleton Board of Education, and extended to the next grade level each succeeding year, so that in 1975-6 it was in operation in grades K through five. Enrolment in the program is optional, at the parents' choice. In grades K (half-day) and one, all instruction is in French (see Barik & Thanks are expressed to the administrators of the Ottawa Board of Education and Carleton Board of Education and to the principals, teachers, and students of the schools involved for their cooperation. Thanks are also addressed to Edna A. Nwanunobi and Sharon Lapkin for the significant roles they played in the research. Requests for reprints may be addressed to either author, Modern Language Centre, The Ontario Institute for Studies in Education, 252 Bloor Street West, Toronto, M5S 1V6. CANAD. J. BEHAV. SCI./REV. CANAD. SCI. COMP., 10(3), 1978 FRENCH IMMERSION IN OTTAWA 193 Swain, 1975, for details). In grade two, English Language Arts are introduced for one hour per day (except in some schools which introduce them in grade three), for an approximately 80% French, 20% English program. The same general formula is followed in grades three to five in the Ottawa Board of Education, but in the Carleton Board a change occurs in grade five, where mathematics is taught for the first time in English, for an approximately 65% French, 35% English program. Both Boards will change to a 50-50 program in grade six and subsequent grades. PROGRAM EVALUATION BACKGROUND At the inception of the French immersion program, both Boards of Education entered into an agreement with the Bilingual Education Project of the Ontario Institute for Studies in Education, which monitors a number of bilingual education programs in Ontario, to have yearly evaluations of the program carried out. These evaluations address themselves to the concerns of the two boards at the start of the program, relating to the levels of achievement attained by the pupils in the program in French and in other areas (English, mathematics, science) relative to pupils in the regular English program, as well as to any effects on mental ability or IQ (see Barik & Swain, 1975, for a more detailed statement of aims of evaluation). The evaluation has considered the three lead groups in the program: Cohort i, which began the program in K in 1970 and was in grade five in 1975-6; Cohort II , which began in 1971 and was in grade four in 1975-6; and Cohort HI, which began in 1972 and was in grade three in 1975-6. In general, previous evaluations of the program (Barik & Swain, 1975, 1977; Swain & Barik, 1976a, see also 1976b) have revealed that: (a) the level of proficiency in French achieved by pupils in the French immersion program is much superior to that of pupils taking French for daily periods of 20 to 40 minutes; (b) pupils in the immersion program lag behind their regular program peers in several English language skills until grade two or three, but catch up soon after English Language Arts are introduced into their curriculum; (c) content material (mathematics) is learned as well via French as via English; (d) mental ability when measured by IQ tests seems to be unaffected by the type of program (immersion versus regular) although a positive effect of the immersion program has been noted in grade four (Barik & Swain, 1977). These findings are subject to slight variations, depending upon the specific group or cohort of pupils being considered. Table 1 gives a longitudinal and replicational overview of findings over the six-year period, K through five, with respect to IQ, English language skills, and mathematics. 194 BARIK & SWAIN TABLE 1 Summary of IQ, English, and Mathematics findings for Cohorts i, n and m in grades K-5, French immersion (/) and comparison (C) groups Grade IQ Cohort K ns i II m English i II Math. / > C** / > C* letter recog.*) 1 ns ns ns 2 ns ns ns C> I: ns C> I: C> I: read.*** tot.read.* spell.*** spell.*** wd.know.*** wd.disc.*** read.*** 3 ns ns ns spell.* C> I: 5 4 / > C*** / > C* ns ns I> C: vocab.** punct.*** usage* lg.tot.** I> C: letters & wd.know.*** wd.know.* wd.know.** vocab.* tot.read.* usage* sounds**) wd.disc.*** read.* OI: capit.* m ns C> I: wd.know.** wd.disc.*** read.*** C> I: spell.* i ns ns ns or. ns prob.solv.* C > /: ns concepts* prob.solv.* II ns m ns ns C> I: capit.*** punct.** lg.tot.** ns ns / > C: ns ns comput.*** Note, IQ findings based on unadjusted scores; English and Mathematicsfindingsbased on data adjusted for age and IQ. From data presented in Bank and Swain, 1975, 1977; Swain and Bank, 1976a; and present article. *p < .05. **p < .01 ***p < .001 1975-6 EVALUATION Sample In contrast to previous evaluations which involved testing in each school of the two Ottawa Boards where the immersion (/) program was in effect at therelevantgrade levels, in 1975-6 the sample was restricted to fourrepresentativeclasses at each grade level (grades 3, 4, 5), together with comparable classes from the regular English program (from either the same or matched schools, on the basis of SES characteristics supplied by the Boards of Education and containing census data relating, e.g., to proportion of single-family dwellings and home ownership), which constituted the comparison (C) group (approximately 95-120 pupils in each program at each grade level). The pupils from both programs can be characterized as generally from a middle to upper-middle socioeconomic background. (The enrolment of FRENCH IMMERSION IN OTTAWA 195 pupils in the immersion program is optional, and hence there is no pre-assignment of pupils to one or the other program. Since complete classes were tested, no pretreatment matching of pupils in the two programs was possible. The pupils are representative of classes in the two respective programs. Potential differences between the two groups on factors such as age and IQ are eliminated in the analysis through analysis of covariance techniques.) AH immersion classes had begun English Language Arts in grade two except for one grade three class which started in grade three. Of the four grade five classes, two followed the mathematics program in French and two in English. As in previous evaluations, some pupils were excluded from the analysis on the basis of several criteria (e.g., pupils with physical problems, recent immigrants with limited ability in English, repeaters), and the scores obtained on tests of French proficiency by pupils from home backgrounds where French was spoken were excluded from the analyses relating to French tests. Test Battery The following test battery (except for c) was administered to all groups, the appropriate level of the test being employed in each instance. Testing was carried out in Spring 1976 (except for item e): (a) Canadian Cognitive Abilities Test, Nonverbal Battery. This test (CCAT) was used to obtain an IQ measure (or "standard age score"). The Nonverbal Battery is intended to measure what has been called "fluid intelligence," that is, ability that is not bound by formal school instruction. (The CCAT replaced the Otis-Lennon Mental Ability Test employed up to grade three in previous evaluations.) (b) Canadian Tests of Basic Skills (Form 3). This test battery (eras) provides for comprehensive and continuous measurement of growth in the fundamental skills: vocabulary, reading, the mechanics of writing, methods of study, and mathematics. The five main tests in the battery are: (1) Vocabulary, (2) Reading Comprehension, (3) Language Skills (spelling, capitalization, punctuation and usage), (4) Work-Study Skills (map reading, reading graphs and tables, and knowledge and use of reference materials), (5) Mathematics Skills (concepts and problem solving). The. scores on all subtests of the CTBS are expressed in terms of grade equivalents, and the scores of the five main tests are averaged to yield a composite score for the total test battery. (The CTBS replaced the Metropolitan Achievement Tests battery employed in previous evaluations.) (c) Metropolitan Science Test (Intermediate). This test was administered to grade five students only, and marked the first time that achievement in science was evaluated. Standard scores were used in the analysis. (d) Story Completion Test. This is a cloze test, with the first and last sentences left intact and every seventh word in the main body of the test deleted, to be filled in by the pupil. All pupils performed this task in English. (/ pupils were also given a similar task in French, but the main analysis there is of a linguistic nature and is reported elsewhere; Lapkin & Swain, 1977.) The story employed (approximately 280 words in length) was written by a specialist in children's literature, and was appropriate in content and style to the grade levels tested. Scores were obtained using the acceptable-response scoring procedure (words that are grammatically acceptable in the context) rather than the exact-response scoring procedure (only words that replicate those in the original; for details, see Swain, Lapkin, & Barik, Note 1). This type of test has been demonstrated to have considerable validity as a measure of overall first language proficiency as well as overall second language proficiency (Oiler & Conrad, 1971; Swain, Lapkin, & Barik, Note 1). In this paper, only the total scores on the 196 BARIK & SWAIN English test are reported; types of errors made by the pupils (in both English and French in the case of immersion pupils) are discussed elsewhere (Lapkin & Swain, 1977). Two other tests were administered to French immersion pupils only: (e) Test de Rendement on Francois is a series of tests of achievement in French used by the Commission des Ecoles Catholiques de Montreal and by the Ministry of Education in Quebec with native French-speaking students, which has been employed in previous evaluations of the French immersion program. The tests consist of items measuring various types of language skills. (The grade three test, for example, involves such topics as synonyms, antonyms, rearranging words into a sentence, alphabetical sequences, tenses, and stylistics.) The tests were administered in late November to correspond to the dates of administration in Quebec. The same test editions as in previous evaluations were used in grades three and four, while the grade five test (1973-4 edition) was employed for the first time. Raw scores were recorded (maximum scores possible of 30, 40, 35 in grades 3, 4, 5 respectively), which were then referred to stanine norms established for native French speakers in Montreal. (f) Test de Lecture is a series of tests of reading comprehension in French developed by the Bilingual Education Project. The test at each grade level consists of a number of passages, each of which is followed by questions about its contents (9 passages and 28 questions in grade three, 10 and 42 in grade four, 8 and 40 in grade five). The grades three and four versions were slightly revised from those employed in previous evaluations, while the grade five test was developed for the present evaluation and was thus employed for thefirsttime. Norms for these tests have not yet been developed. (Writing skills in French and English in the case of/ pupils and in English only in the case of C pupils were also considered, but lie outside the scope of this presentation. See Swain, Note 2, for previous evaluation.) RESULTS The data for all three grade levels are shown in Table 2. In line with previous evaluations, differences between / and C groups were analyzed through one-way analysis of covariance, covarying out the effects of age and IQ in comparisons relating to other variables, i.e., CTBS, Science, and English story completion test scores. (Even though findings over the years have generally revealed little difference between the two groups in relation to the two covariates, their use is judged appropriate. Age is employed as a covariate since at early grade levels age variations may result in maturational differences affecting test performance, IQ is used as a covariate since it bears a general relationship to test performance. The rationale for using the current year rather than pre-program IQ measure as the covariate has been presented by the authors previously, the argument being that to the extent that IQ affects performance on achievement tests, it is the IQ measure concurrent with that testing that should be employed as the covariate to equate the two groups on that variable, while the possible effect of the type of program - immersion versus regular - on IQ as such is an independent issue (see Swain & Barik, 1976b, p. 47, note 5). There is, in fact, no pre-program IQ measure, since the immersion program begins in kindergar- FRENCH IMMERSION IN OTTAWA 197 ten, with no pre-entry testing. A previous evaluation, however, showed no reliable IQ difference between / and C pupils towards the start of the K year (Bank & Swain, 1975). The individual was used as the unit of analysis, rather than the class as had been the case in previous evaluations, owing to the reduction in the number of classes tested. In relation to all grade three group comparison analyses except that pertaining to IQ, the immersion class which began English Language Arts instruction in grade three instead of grade two was excluded, since reading in English was required in all instances, putting that class at a disadvantage relative to the others. (However, an analysis of that one class versus the other grade three immersion classes which began English Language Arts in grade two revealed little difference on any of the variables.) IQ There are no differences in the IQ scores of/ and C pupils in grades three and four (Cohorts m and 11), but there is one favouring/ pupils in grade five (Cohort i). The findings for each cohort correspond to those observed the previous year in grades 2,3, and 4 respectively (Bank & Swain, 1977). (The longitudinal effects of the immersion program on IQ have been examined in detail in Bank & Swain, 1976.) English Language Skills In grade three (Cohort m), / pupils are behind C pupils in several English language skills: capitalization, punctuation, and language total, which combines these two variables with spelling and usage. The previous year in grade two, / pupils were found to be behind in relation to spelling on the basis of a different test. In terms of replications, the trend of the data for Cohorts i to HI in grade three indicates the / pupils at that grade level lag somewhat behind in the "technical" aspects of English language skills (spelling, punctuation, capitalization). The grade 4 (Cohort II) data show the / group ahead of the C group in vocabulary and usage, and behind in capitalization. This represents an amelioration over their grade three performance, where / pupils were behind in several areas. The findings also differ somewhat from those obtained with Cohort i in grade four, where no reliable difference between/ and C pupils was noted on the basis of adjusted scores. The grade five (Cohort i) data show the / group ahead of the C group in vocabulary, punctuation, usage, and language total, and thus also represent an improvement over their previous year's performance in grade four. It is to be noted that for all three grades, the grade equivalent scores of both groups are generally close to or above their own grade placement at TABLE 2 Adjusted mean scores for grades 3, 4, 5 Grade 4 Grade 3 JV(max.) Age (mos., end yr.) Cdn. Cog. Ab. T. (Stand. Age Sc. (= IQ) Cdn. T. Bas. Sk.,' Grade equivs. Eng. Skills: (a) Vocab. (b) Read. Comp. (c) Spelling (d) Capitalization (e) Punctuation (f) Usage (g) Lang. Tot. (c-f)b Math Skills: (h) Concepts (i) Prob. Solv. (j) Maths. Tot. (h-i)b Grade 5 Immersion group Comparison group Immersion group Comparison group Immersion group Comparison group 104 107.81 75 106.96 96 119.58 68 119.21 83 131.29 101 130.03 105.62 105.16 110.12 107.38 113.00* 108.22 4.31 4.19 4.04 3.58 3.99 3.94 3.89 4.36 4.43 4.18 4.31*** 4.43** 4.20 4.25** 5.51* 5.21 5.09 4.93 5.24 5.37* 5.16 5.19 5.02 5.26 5.32* 5.15 4.97 5.19 6.59** 5.95 6.01 5.99 6.41*** 6.27* 6.17* 6.13 5.87 6.04 5.64 5.81 5.91 5.86 3.96 4.06 4.03 4.02 4.01 4.04 5.13 4.98 5.08 4.98 4.82 4.93 6.18 6.00 6.11 6.25 5.96 6.14 TABLE 2 (concluded) Grade 3 Immersion group Work Study Skills: ) Map Reading %1) Graphs & Tables (m)) Use Ref. Mat. (n) Wk. Study Tot. (k-m)b (a, b, g, j , n)b Eng. Story Compl. T." (max. = 30) MAT Science T., Stand. Sc." (range 55-109) Test Rend. Fr. (max. = 30/40/35) Test de Lect. (max. = 28/42/40) Grade 4 Comparison group 4.17 4.00 3.85 4.03 4.12 4.06 3.78 4.00 4.12 20.86 4.22 20.54 16.39 20.19 — Immersion group 5.30* 5.43* 4.99 5.25* 5.26 22.86 23.39 25.16 Grade 5 Comparison group Immersion group Comparison group 4.97 5.02 4.70 4.90 6.10 6.17 5.93 6.07 6.11 6.04 5.78 5.98 5.05 22.88 6.17 24.43 5.99 24.36 — 84.58 17.06 19.24 84.84 "Scores adjusted for age and IQ (unadjusted scores fractionally different from those presented here: maximum unadjusted-adjusted (UA) CTBS score difference = .02 in grade 3, .04 in grade 4, . 12 in grade 5; maximum UA English Story Completion Test score difference = .05 in grade 3, . 13 in grade 4, .27 in grade 5; maximum UA Science score difference = .89 in grade 5. Statistically significant findings on basis of one set of data (u or A) but not other as follows: in grade 3, no differences between the 2 sets; in grade 4, p < .05 for CTBS use of reference materials with u scores; in grade 5, p < .05 for CTBS spelling and p < .01 for punctuation and composite score with u scores). "Not exact average of subtest scores due to occasional missing subtest data. *p S, .05. **p <, .01. ***p < .001. 200 BANK & SWAIN the time of testing (3.8,4.8, 5.8 for grades 3, 4, 5 respectively). This is also the case in relation to the other sections of the CTBS . As for the English story completion test, which provides a general measure of language competence, no differences are noted between the two groups throughout grades three to five. Mathematics Skills. No differences are found in mathematics at any grade level. (In grade five, the / data of the two Boards have been combined since a separate analysis reveals no difference between pupils taught in French and those taught in English.) These findings correspond to those of the previous year with the same cohorts except for Cohort n, where in grade three the / group scored lower than the C group on a different mathematics test, and are in general agreement with mathematics results over the years. Work-Study Skills and Composite Score. There are no significant differences in work-study skills in grades three and five, while in grade four the / group scores significantly higher on two of the subtests and on work-study total, even though it was asked to do tasks in English which would normally be performed in French in school. As for the composite score on the CTBS, no differences are noted at all three grade levels, and it can thus be concluded that the French immersion pupils are keeping up with their peers in the regular English program in relation to the spectrum of academic skills acquired in the course of their schooling, even when the testing situation is somewhat biased against them by having them tested in English for skills acquired in French. Science. No reliable difference is found between the two grade five groups in science achievement tested in English. (/ pupils are taught the subject in French.) French Performance. On the Test de Rendement en Frangais,I pupils at all three grades score in the stanine four range relative to norms for native French-speaking pupils in Quebec, thus performing as well as from 24 to 40% of the French population. This finding is generally consistent with previous evaluations. On the Test de Lecture, the results cannot be gauged against test norms, but the scores obtained are generally consistent with those of previous years where applicable and are judged satisfactory by French immersion program consultants. They also tend to be somewhat higher than scores obtained on an experimental basis by small groups of French or bilingual background pupils in a Franco-Ontarian school, though these comparisons have to be treated with caution because of uncontrolled factors (e.g., different socioeconomic conditions). In general, however, performance in French at all three grade levels appears satisfactory. The results of the 1975-6 evaluation for the primary grade French immersion program in operation in the Ottawa area public school system FRENCH IMMERSION IN OTTAWA 201 confirm previous findings that pupils in the program are not showing any long-term lag in academic development relative to regular program pupils, and are, in fact, ahead in several respects, while continuing to demonstrate a satisfactory level of performance in French. Thesefindingsare in general agreement with those from similar investigations in other parts of the country. Evaluation du programme d'apprentissage du francais par immersion implante dans les ecoles du systeme public d'enseignement de la ville d'Ottawa. Cette evaluation, faite dans les classes comprises entre la troisieme et la cinquieme annees, indique que, dans la plupart des matieres scolaires (anglais, mathematiques, techniques de travail, sciences), les ecoliers impliques dans ce programme ont un rendement en general equivalent ou superieur a celui des ecoliers de meme niveau qui suivent le programme regulier d'enseignement en anglais, et reussissent par ailleurs de facon satisfaisante en francais. Ces resultats confirmerit ceux qu'avaient fournis d'autres evaluations. REFERENCE NOTES 1. Swain, M., Lapkin, S., & Barik, H.C. The cloze test as a measure of second language proficiency for young children. Working Papers on Bilingualism (No. 11,32-43). Toronto: Ontario Institute for Studies in Education, 1976. 2. Swain, M. Writing skills of grade three French immersion pupils. Working Papers on Bilingualism (No. 7,1-38). Toronto: Ontario Institute for Studies in Education, 1975. (Reprinted in Swain & Barik, 1976b). REFERENCES Barik, H . C , & Swain, M. Three-year evaluation of a large scale early grade French immersion program: The Ottawa Study. Language Learning, 1975,25,1-30. Barik, H . C , & Swain, M. A longitudinal study of bilingual and cognitive development. International Journal of Psychology, 1976, / / , 251-263. Barik, H.C., & Swain, M. French immersion in Canada: The Ottawa Study through grade four. ITL, A Review of Applied Linguistics, 1977, no. 36,45-70. Bilingual Education Project. French immersion programs in Canada. Canadian Modern Language Review, 1976,32,597-605. Lambert, W.E., & Tucker, G.R. Bilingual education of children: The St. Lambert experiment. Rowley, Mass.: Newbury House Publishers, 1972. Lapkin, S., & Swain, M. The use of English and French cloze tests in a bilingual education program evaluation: Validity and error analysis. Language Learning, 1977,27,279-314. Oiler, J.W., & Conrad, C.A. The cloze technique and ESL proficiency. Language Learning, 1971,27,183-196. Swain, M. Bibliography: Research on immersion educationfor the majority child. Canadian Modern Language Review, 1976,32,592-596. Swain, M., & Barik, H.C. A large scale program in French immersion: The Ottawa Study through grade three. ITL, A Review of Applied Linguistics, 1976, No. 33, l-25.(a) Swain, M., & Barik, H.C. Five years of primary French immersion: Annual Reports of the Bilingual Education Project to the Carleton Board of Education and the Ottawa Board of Education up to 1975. Toronto: The Ontario Institute for Studies in Education, 1976.(b) First received 14 March 1977