Standardized testing: Misused and misunderstood

The Ontario Ministry of Education touched off some controversy last spring when it launched the School Information Finder website. This site made it easy for parents to school shop online by comparing schools’ provincial test results and other information, such as percentages of low-income families, ESL students, even parents with university education. Browsers could select schools and put them in a “school bag,” kind of like the shopping carts on commercial Internet sites.

The initial wave of protest from parent and teacher groups was so fierce that the ministry removed the school bag function within days of launching the site.

I abhorred that function too. But the bigger issue for me is the way this website and other influences, such as the news media, the Fraser Institute and the C.D. Howe Institute, which rank schools in some Canadian provinces based on standardized test “scores,” are pushing parents in the direction of thinking school test results mean a lot more than they actually do.

All Canadian provinces and territories have provincial tests, and all make the results public, as they should. But promoting schools’ test results on a website designed to facilitate school shopping sends the message that these numbers indicate something important, an idea that doesn’t stand up to scrutiny. Test results are most useful for diagnosing problems with curriculum delivery in a school district or region. That’s what they were designed for, not comparing schools based on the test scores from three or four grades three and six classes.

Even if you think comparing schools by test data is useful, which I don’t, last year’s results (which are what the Ontario site offers) mean very little as a way of assessing a school, and are totally meaningless in terms of predicting how your child will fare at that school. My kids went to an elementary school that is consistently in the bottom 11 percent of the Fraser Institute’s rankings. Do I think my son who struggled in school would have somehow found it easy to pay attention and complete assignments at a higher-ranked school? Do I think that my son who was identified as gifted would have been “more gifted” at a “better” school? Hardly.

Don Klinger, a professor of education at Queen’s University in Kingston, Ont., who, by the way, believes that provincial tests have value, told me that factors related to a school itself — quality of instruction, school policies, teachers — account for only 10 to 25 percent of the difference between kids on large-scale tests. Most of the difference — 75 to 90 percent — is due to factors related to individual kids: family, income level and a student’s attributes, such as intelligence or motivation. “The quality of teaching doesn’t vary that much in Canadian schools,” Klinger says. “Yes, there are exceptions — outstanding schools or teachers who are particularly good at supporting learning. Clearly, schools and teachers matter, but the large-scale assessments we currently have are not particularly good ways of identifying what it is about schools and teaching that contributes to student achievement.”

The best use of test scores might be to identify schools that need extra support — although any superintendent who can’t figure that out without provincial tests should be bused back to teacher’s college. As for parents, do what you wish with test scores. Just be careful of what you think you know. Standardized test results tell us much less than some people want us to believe.