Flawed report, flawed conclusion

Published Monday April 21st, 2008
A5

The recent decision to terminate early French immersion in New Brunswick is unwelcome news for many New Brunswickers.

Caption
Noel Chenier/Telegraph-Journal
Surrounded by his fellow government members, Education Minister Kelly Lamrock tries to speak to a large crowd gathered at the New Brunswick Legislature in Fredericton to voice their opposition to the government's plan to eliminate early French immersion.

The centerpiece of Minister Lamrock's decision was the Croll and Lee report, released in late February. Mr. Lamrock quickly indicated that he liked what he saw in the report. Others were confused; this represented a radical change in direction from previous reports, which called on the government to fix early immersion, not terminate it. So, is this change justified?

We, and a growing number of others, think not, for a variety of reasons. First, the government is ignoring French Second Language (FSL) experts, all of whom have come out against this decision. Dr. Croll and Ms. Lee are not FSL experts. Second, we found many fundamental mathematical flaws that undermine the core recommendations of the report. Following are some of the major ones:

*Attrition - Croll and Lee stated that attrition, or "drop out" rates in early immersion are much higher than they are in late French immersion. They refer to persistence in late immersion as a "comparative success" in this regard. However, these rates were calculated incorrectly throughout the document, and proponents of the report have recently acknowledged that their calculations did not represent attrition rates at all. When calculated properly, there is virtually no difference in attrition between the two programs, and annual attrition (divided over the life of each program) is somewhat lower in early immersion than late immersion.

*Success rates - Success rates of the various programs were underestimated in the report. Students at the end of FSL programs may take an optional Oral Proficiency Exam. Croll and Lee assumed that anyone who did not take the exam had learned no French. Further, a significant number of immersion students do drop out late in high school because courses that interest them are not available in French. These students are not tested, and again were assumed to have failed in the program, creating a downward bias in results and ignoring the fact that many of these students speak excellent French. The report therefore measured success as receiving a certificate rather than learning French.

To understand how much French is learned in each of these programs, we used the Croll-Lee data to calculate how many students achieved the various levels of French proficiency. We found that although fewer students were tested in early immersion than in late immersion, more early immersion students attained intermediate plus and advanced standards. If the intensive French program works as planned, it is possible (though not certain) that more core students will reach the intermediate level, but it is highly likely that there will be many fewer students reaching both intermediate plus and advanced levels. Hence, as FSL experts have stated, this change represents a significant lowering of the bar.

*Costs - Contrary to what the report suggests, early immersion is not sapping resources. We used the Croll-Lee cost calculation method and looked at how much it costs to reach a set level of proficiency. Early immersion is more cost effective at helping students to reach both intermediate plus and advanced levels. Further, if you look at the percentage of French language funding devoted to the three programs (core, early immersion, late immersion) relative to the number of students in each, late immersion costs about 30 per cent more than early immersion on a per student basis. Support for the core program clearly needs to increase, but on a financial basis it should not be coming at the cost of dismantling early immersion.

*Streaming - Recently, Mr. Lamrock has focused on how this change will end the problem of ability-based streaming in our schools. There is no doubt that most struggling or special-needs children are in the English stream. That is regrettable, and occurs because the early immersion program lacks the resources needed to help these students, and parents have been discouraged from placing them in immersion. Mr. Lamrock has concluded that if immersion goes away, the number of challenged children in each class will drop. Further, he contends that eliminating streaming will help improve our dismal provincial test scores by providing a more positive learning environment for all children. Data from the Croll-Lee report do not support either of these assumptions.

First, based on enrollments in 2005, and assuming an average Grade 2 class size of 23 (taken from the government's website), there is a province-wide average of 5.43 children with special needs per English class. If early immersion and English classes were merged, that average would drop to 4.25 - a reduction of 1.18 children per class, not exactly the massive reduction anticipated.

Second, Croll and Lee are concerned that streaming is partially responsible for poor achievement on standardized tests. However, using their own data, in Grade 2 in 2006-07, core English students scored an average of one point higher on the provincial reading assessment than did early immersion students.

By Grade 4, the situation changed only slightly, with early immersion children averaging 2 per cent higher than core English children. Further, at that point, slightly more (69.5 versus 66.8 per cent) core children met the provincial standard achievement level than did early immersion children. It's hard to see any evidence of streaming in grades 2-4.

By Grade 5, early immersion children beat core children by about 5 per cent in math. However, the variation in scores is very large, and the model they presented indicates that program differences explain only 1.2 per cent of total variation in results. Thus, something other than streaming is responsible for 98.8 per cent of the variation in test scores. This is extremely weak evidence on which to base a streaming argument.

Finally, by Grade 8, effects of streaming on math scores do appear to be substantial. Students in English do the worst (53.7 per cent), those in early immersion are in the middle (64.9 per cent), and those in late immersion do the best (68.2 per cent).

Croll, Lee, and Minister Lamrock seem to miss the important point that streaming only becomes a substantial issue AFTER late immersion students are separated out from the rest.

In other words, they offer no evidence that elimination of early immersion will fix the streaming problem in later years, and these data suggest exactly what Dr. Joe Dicks has said publicly - that having only late immersion may produce an even bigger streaming problem than we have now.

In conclusion, all of this is not "quibbling over numbers," as Minister Lamrock has said. The errors identified in the Croll and Lee report are so substantial that they undermine both its recommendations, which the Minister has adopted, and public confidence in its objectivity. N.B. taxpayers paid $180,000 for this effort, and we deserve an explanation of how it supports the Minister's decision.

Minister Lamrock has stated that his job is to help all children. We fully agree, and recognize that changes need to be made to the core French program. However, the decision to end early immersion does not help all children; it harms those who choose early immersion. He has seemingly recognized this, stating that he had to make this choice at the expense of "20 per cent" to avoid leaving "80 per cent" of children behind. The 20-80 argument is inaccurate. The correct percentage in immersion classes ranges from about 32 to 24 per cent in elementary grades, increases to near 40 per cent in middle school as late immersion students join, and drops in high school as students (now bilingual) leave the program. Hence, the cost to children is greater than he implies.

Since the Minister says this is not about money, why not invest funds to improve core French (e.g., offer intensive French when appropriate), keep early immersion, and invest resources to make it more inclusive. That way, everyone benefits, nobody is harmed, and possible negative effects of streaming are avoided. This offers the best chance of reaching the goal of 70 per cent bilingual graduates. The minister has stated that French-speaking resource people do not exist and cannot be trained in time to solve the problems associated with lack of inclusion. Others have disagreed with him on this.

The bottom line is that there is no support among FSL experts for this decision, and the Croll-Lee report does not deliver the required evidence.

Minister Lamrock has stated that his job is to "look at reports - all the information - and make a decision on what's best for all kids." Did he consider Rehorick et al. (2006), written by FSL experts and recommending strengthening early immersion to make it more inclusive? Further, both the Scraba (2002) and MacKay (2006) reports identified problems and recommended changes to improve inclusion, but neither recommended eliminating early immersion, and both opposed a one-size-fits-all approach.

Given the wealth of information provided in these and other studies, why did the Minister adopt the recommendations of the flawed Croll-Lee report? Why rush into an unproven system?

The Graham government is gambling with our children, and they need to demonstrate to citizens that this is a good wager. Simply telling us that everyone who disagrees with them is biased for some reason or another and therefore should not be considered isn't cutting it.

With the weight of expert evidence so heavily skewed against them, we should all be demanding an explanation of what is going on in education in this province.

Diana Hamilton is an assistant professor at Mount Allison University. She teaches advanced statistics to biology students. Matthew Litvak is a professor at UNBSJ. He has also taught advanced statistics in biology. They have reviewed the analysis found within the Croll and Lee report. Their review can be found at http://hamlit2008.googlepages.com

Please Log In or Register FREE

You are currently not logged into this site. Please log in or register for a FREE ONE Account.
Logged in visitors may comment on articles, enter contests, manage home delivery holds and much more online. Your ONE Account grants you access to features and content across the entire CanadaEast Network of sites.

Comments (46)

All comments are subject to the site Terms of Use. For a full commenting tutorial click here.

Our editorial team relies on filtering technology and our visitor community to identify inappropriate comments. In the event that a site user has submitted offensive content that has evaded our filter, please select the option to Flag As Inappropriate presented within the comment. Thank you for helping to keep this site clean.

I agree, although the arrogrant Min of Education just thinks I am just an emotional parent!

Great editorial.
91
Thumbs Up
12
Thumbs Down
Flag as Inappropriate
Flag as Inappropriate
Anonymous Reader on 21/04/08, 6:26:46 AM ADT
Thank you, professors. Unfortunately, the Minister is not interested in what experts, parents, teachers, and most of all, students, have to say. Let's keep making our voices heard until 2010. This is SO wrong!
89
Thumbs Up
10
Thumbs Down
Flag as Inappropriate
Flag as Inappropriate
Anonymous Reader on 21/04/08, 7:09:13 AM ADT
A clear and reasoned analysis of the problem. And not the least bit emotional!
92
Thumbs Up
11
Thumbs Down
Flag as Inappropriate
Flag as Inappropriate
Anonymous Reader on 21/04/08, 7:13:43 AM ADT
This report is the only one that ever said to eliminate EFI - Lamrock keeps saying there have been 5 reports over 15 years - true. But all the other ones said to strengthen it, offer Methods and Resources in EFI (which has never been done).

This report is the only 'official' basis for this decision. And its completely wrong.

So, then where did the real plan come from? No one in Government has answered that. I suspect its not something they want the public to know.


82
Thumbs Up
10
Thumbs Down
Flag as Inappropriate
Flag as Inappropriate
Anonymous Reader on 21/04/08, 7:46:18 AM ADT
Thank you for a reasoned and intelligent response. Much wants more. Please look at the PISA tests and the exaggerated claims made about illiteracy in the province and the unfair comparisons with other provinces. It would be so nice to see this done by someone who does not have a political or corporate agenda.
83
Thumbs Up
10
Thumbs Down
Flag as Inappropriate
Flag as Inappropriate
Anonymous Reader on 21/04/08, 7:59:45 AM ADT
Yes, how about the fact that NB is the only province who includes students with special needs in their PISA scores.
71
Thumbs Up
7
Thumbs Down
Flag as Inappropriate
Flag as Inappropriate
Anonymous Reader on 21/04/08, 8:34:46 AM ADT
Excellent article! Hopefully more people will now become aware that the recommendations put forth in the Croll/Lee report are not supported by any proper analysis. I'm hopeful that given your thorough explanations of the real statistics associated with Attrition rates, Success rates, Costs and Streaming, more people will realize that this is not a rational decision and raise their voices in opposition.

73
Thumbs Up
9
Thumbs Down
Flag as Inappropriate
Flag as Inappropriate
D. Doucet, Fredericton on 21/04/08, 10:03:46 AM ADT
Misery loves company a I guess.....Let's move on folks, streaming will end in New Brunswick and there will be equality for all, not just for the elite.
11
Thumbs Up
87
Thumbs Down
Flag as Inappropriate
Flag as Inappropriate
Anonymous Reader on 21/04/08, 10:04:16 AM ADT
"With the weight of expert evidence so heavily skewed against them, we should all be demanding an explanation of what is going on in education in this province."

This doesn't seem so unreasonable. If the Minister has evidence that proves eliminating EFI will significantly reduce streaming and increase literacy levels why can't we see it. With the exception of Croll Lee, which he has now distanced himself from, I have seen no attempt to explain/justify how the changes will achieve this.

Believe me if the evidence showed up I could easily be convinced to buy into it. As it stands however, I have trouble with someone telling me I'm right because I'm me and I know what's best because I said so.

If it's about money--say so. If not let us see the evidence.

67
Thumbs Up
8
Thumbs Down
Flag as Inappropriate
Flag as Inappropriate
Anonymous Reader on 21/04/08, 10:05:32 AM ADT
Anon 10:04 - Did you even read the article? With this plan, streaming will NOT end in NB, it will be delayed, but it will likely increase!
60
Thumbs Up
9
Thumbs Down
Flag as Inappropriate
Flag as Inappropriate
Anonymous Reader on 21/04/08, 10:37:18 AM ADT
Advertisement
Advertisement

Search Articles