Somebody get me a 15-year old to explain these.

December 4, 2007 4:39 AM

PISA results are in.Finland and Canada high five and make awkward polite bows and gestures towards The Far East. For education policy Programme for International Student Assessment is the research about how 15-year old students are being educated. Briefings for UK and US. Results escape easy answers, but you might test your theory against interactive data tool.

(Site might be slow today and I'm having difficulties reading the full pdf:s with Apple Preview. Has some strange disclaimers covering that won't react to clicks. Probably needs Adobe stuff. Data tool is (smug) fun.)

posted by Free word order! (47 comments total)

Finland and Canada? Don't kids in cold climates have anything better to do than to read books?
posted by three blind mice at 5:19 AM on December 4, 2007

Who needs math and science when you've got Jesus™?
posted by Mister_A at 5:49 AM on December 4, 2007

I'm not 15 but:
US performs poorly in science (and reading and math) on OECD assessments. Scores are more variable than in other countries: we have as many people or more at the very high end of achievement but many more at the low end. This would seem to point to great inequalities in terms of educational opportunities.

Finland, Australia, Canada, Korea also scored highest on the last OECD-PISA report on literacy.

Of course there are many factors combining to yield these outcomes: relative economic, ethnic, and linguistic homogeneity seems important. We clearly have a greater spread of incomes and opportunities in this country, though.
posted by cogneuro at 6:02 AM on December 4, 2007 [2 favorites has favorites]

I meant to say greater inequalities in terms of opportunities here than elsewhere.
posted by cogneuro at 6:04 AM on December 4, 2007 [1 favorite has favorites]

Cognero, you're saying Canada has linguistic homogeneity?
posted by Space Coyote at 6:11 AM on December 4, 2007

Yes, that's exactly what he's saying, space coyote. What a fantastic "gotcha".
posted by Mister_A at 6:15 AM on December 4, 2007

I was very disappointed that the data tool only gave access to the questionnaire data and not the assessment data. Still the US report seems to be getting at an important point: when it comes to the amount getting the highest level we're doing fine. There are other distributional features that I'd like to see :(
posted by a robot made out of meat at 6:39 AM on December 4, 2007

Of course there are many factors combining to yield these outcomes: relative economic, ethnic, and linguistic homogeneity seems important. We clearly have a greater spread of incomes and opportunities in this country, though.

I submit that teacher qualification is the biggest factor of all.

Instead of debating over "merit pay"schemes, the Finns decided that all teachers merit higher higher base pay. To justify this, Finland requires that all primary educators have advanced degrees and they pay them salaries competitive with other industries. This way, they attract talented people into teaching... and then put them into well-equipped rooms with small class size.

Instead of testing students, I say test the teachers, fire those who don't make the cut and pay the ones that do better salaries so they have an incentive to stick around.
posted by three blind mice at 6:43 AM on December 4, 2007 [4 favorites has favorites]

I don't buy language (and by implication, culture) at all. If you go to the last link in the post and sort for language factors:

Canada:
First national lang: 82.99%
Second national lang: 2.24%
other: 10.07%
(not reported 4.7%)

US:
First national lang: 87.63%
other: 10.48%
(not reported 1.89%)

(cf: OECD avg: national lang 90%, other 5%)

Canada may be more homogeneous on a socio-economic level than the US, but cuturally and liguistically, we're diverse. We accept immigrants at twice the per capita rate, after all.
posted by bonehead at 6:46 AM on December 4, 2007

Canada may be more homogeneous on a socio-economic level than the US

Probably because we let them get married.
posted by srboisvert at 7:01 AM on December 4, 2007 [2 favorites has favorites]

I'm not surprised with the current US administration's anti-science policies and activists pushing fake science & creationism in schools.
posted by mike3k at 7:16 AM on December 4, 2007 [1 favorite has favorites]

I know it's fun to harsh out on public school teachers, but the fact remains that they have a high-stress, relatively low-paying job with minimal possibility for advancement, and, in Philadelphia anyway, they are worn down quickly by the never-ending parade of kids with no social skills, no self-control, no parental involvement, and no academic skills.

There are kids entering kindergarten who literally don't even know what a book is. They do not understand that there is a relationship between printed characters and spoken language. THat's a huge gulf that must be crossed before these kids can learn to read, and consequently, you have first and second graders who are illiterate despite having been in public education for a few years. Many of these students never catch up, because there is no incentive to do so. The kids' families place no value on education, their peers often ostracize kids who seem "smart", and you just can't tell a 7-year-old that if he applies himself a bit more, he can get into Temple someday and work in an ad agency or what have you. That has to come from the parents, it doesn't, and the kids fail. Try to understand the miserable situation our public school teachers are in before writing them off as incompetent.
posted by Mister_A at 7:18 AM on December 4, 2007

Testing the teachers isn't as easy as it sounds, either. Though it is imperative that teachers know their subject matter, obviously, that knowledge does not guarantee the ability to teach the subject matter effectively. In other words, some of the best teachers aren't Ph.D.'s.
posted by misha at 7:48 AM on December 4, 2007

great title for the post, btw.
posted by misha at 7:53 AM on December 4, 2007

Yeah, I knew that issue would come up. Basically, yes. There are two main languages but either people are bilingual (which is good) or the languages are segregated regionally. So, in effect, the communities are pretty homogeneous.

These are relative statements, obviously. Countries like Finland, Japan, and Korea are the clearer cases, I think. They are enthnically more homogeneous, linguistically and culturally more homogeneous, and so some of the issues we deal with here aren't as prominent there.

Now, these are all broad generalizations, and I said that many factors are involved. But, I think that this captures something relevant about what's going on. That and the fact that scores are more variable in the US than in any other country (at least they were on the last reading tests; have to see if that's true for the science ones but it's a good bet).
posted by cogneuro at 7:53 AM on December 4, 2007

We have (an) extremely powerful teachers' union(s) in Canada, at least in Ontario. (I'm really not sure about other provinces, but I think it's the same elsewhere) This, I think, contributes somewhat to better quality schools and things like lower class sizes. Sure, everyone wants smaller class sizes - except the politicians paying the bills - but not every country has a extremely powerful lobby pushing for it.

While I like to think that we Canadians score better on these kinds of tests because of our inherent nobility or somesuch, it always comes down to politics in the end IMO.

Anyway, as for being more ethnically homogenous, all I can say is "heh". In 2005 the Toronto District School Board (TDSB) said "For secondary students... 45% had a mother tongue other than English" (p. 19). Hardly homogeneous.
posted by GuyZero at 8:05 AM on December 4, 2007 [1 favorite has favorites]

So, in effect, the communities are pretty homogeneous.

Again, the most polite thing I can say to this is "heh".

You have no idea what a Canadian high school looks like. No. fucking. idea.
posted by GuyZero at 8:07 AM on December 4, 2007

My sister-in-law teaches elementary school. A couple of years ago, in a class of 20-25 kids, she had more than 12 boys named some variant of Mohammed.
posted by bonehead at 8:16 AM on December 4, 2007

"Vancouver School District (#39), the second largest in BC, has a diverse multilingual and multicultural student population. Of its 58,000 students, 61% speak more than 100 different languages other than English at home. "

I could probably pull up similar ESL stats for every English-language school board across the country. Canada's high reading performance may have something to do with the fact that there are aggressive supplemental reading programs in every large school board precisely because there are so many ESL students.
posted by GuyZero at 8:17 AM on December 4, 2007

Here are some other data suggesting my conjecture about linguistic homogeneity is probably wrong. The measure they use is the probability that any two randomly selected people have different languages. On this measure, Canada is more linguistically diverse, and Finland, Japan, and South Korea less so. So, on that measure I'm dead wrong. Data seem a little weird though; why would the probability of agreement be really low in South Korea??

I still think there is something about variability within the culture--extent to which people share values, income variability, etc.--is probably relevant. (Maybe shared language background isn't essential to achieving this.) Because of the level of agreement in some of these countries, there can be things like national curricula that everyone follows, for example. These are interesting research topics.
posted by cogneuro at 8:22 AM on December 4, 2007

Why should Americans be upset if our science scores are low?

We can import most of our scientific talent from abroad anyway.

/tongue-in-cheek>
posted by jason's_planet at 8:29 AM on December 4, 2007

What can I say, Canada is a progressive country and a wonderful place. My understanding is that you have relatively centralized decision making about curricula (I got this impression from talking with one of the officials involved). Whereas we have more people,more states, and a strong tradition that education should be controlled locally. Net result: Texas will be "neutral" about evolution vs. creationism. And how subjects like reading, math, and science are taught is all over the place.
posted by cogneuro at 8:50 AM on December 4, 2007

Cogneuro, if you want an idea of what a Canadian high school looks like, (at least in Toronto), have a look at the videos on the OECD TV page. Cheezy, but more or less accurate in my books.

Areas outside of Pacific BC and Southern Ontario will have a much more homogenous makeup.
posted by anthill at 8:53 AM on December 4, 2007

Actually, I lived in Canada for 11 years. I have some idea.

Let's turn the question around: why is Canada so successful given this diversity?
posted by cogneuro at 8:59 AM on December 4, 2007

My understanding is that you have relatively centralized decision making about curricula (I got this impression from talking with one of the officials involved). Whereas we have more people,more states, and a strong tradition that education should be controlled locally. Net result: Texas will be "neutral" about evolution vs. creationism. And how subjects like reading, math, and science are taught is all over the place.

You left out the single most important repercussion of America's local control approach: funding. Canadian public schools do not rely primarily on local property taxes or other local funding for their survival and health. Schools are funded more or less equally across regions and provinces. A recent immigrant in, say, one of Scarborough's rougher precincts will get roughly the same education as the kids at largely upper-class Leaside Secondary in North Toronto.

In my experience (I was a product of public and Catholic schools, the latter of which are pseudo-public in Canada), even the private school kids whose parents had spent $10,000-plus per year on their education didn't show up at university any better prepared than us middle-class kids. (The biggest difference, as I recall, is their schools tended to have much more elaborate extracurricular programs - wilderness survival outings, debating teams, school trips to Europe.)

On preview: cogneuro, I'd argue this funding thing is one big part; the absence of an oppressively assimiliationist national mythology and the wholesale embrace of multiculturalism as a core value at both the official and everyday social levels helps too.

Haven't read it yet, but pollster Michael Adams' new book Unlikely Utopia addresses some of this stuff at the demographic/public opinion level. And he's an engaging writer as far as pollsters go.
posted by gompa at 9:07 AM on December 4, 2007

What can I say, Canada is a progressive country and a wonderful place.

Let's not confuse my jingoism with any real attributes of Canada.

Let's turn the question around: why is Canada so successful given this diversity?

You can make a successful career in Canadian journalism or politics of either supporting or opposing this question alone.

There was some sort of recent study on this exact topic, but my memory completely fails me on where I read about it.

On the specific topic of reading scores in standardized tests, I go back to ESL programs. The government dumps a lot of money into ESL program both for adults and in the school system. Many school boards have their own standardized testing regime - again, as a life-long Ontario resident (sad but true), I can really only speak to that, but there are standardized reading tests in grades 3 and 6 province-wide.

With these two factors, the school boards have a lot of resources devoted to teaching kids to read, with extra effort for those who would otherwise do the worst, kids who speak/read english as a second language.

The issue is similar to the the hybrid-SUV thread: you get the biggest testing gains by raising the scores of the poorest performers. The Ontario educational system does just that.
posted by GuyZero at 9:14 AM on December 4, 2007

Also, it looks like I have a bad habit of turning any thread tangentially related to Canada into a thread about the minutiae of Canadian life. Apologies.
posted by GuyZero at 9:16 AM on December 4, 2007

gompa: seems like a big part of the equation.
anthill: TO is not all of Canada. What's the linguistic diversity in Sudbury (seriously)?

I don't work up full research reports before posting! These are complex questions and it's interesting to argue about them even if all the data are not in! I'll try not to say anything too stupid too often!
posted by cogneuro at 9:17 AM on December 4, 2007

GuyZero hit upon a very important point:
you get the biggest testing gains by raising the scores of the poorest performers. The Ontario educational system does just that
Yes this is true. However, we, in my opinion don't treat our high achievers as well as some other countries (such as the UK).
posted by niccolo at 9:31 AM on December 4, 2007

Finland apparently broke the point record for the Pisa-study.
*does a little dance*

Just like the Eurovision with Lordi.

What?
posted by slimepuppy at 9:32 AM on December 4, 2007 [1 favorite has favorites]

anthill: TO is not all of Canada. What's the linguistic diversity in Sudbury (seriously)?

Sudbury's probably a good deal less diverse than Toronto - I went to high school in North Bay, a couple hours east, and there were at most a dozen non-white faces in my graduating class of 300 - but that's mainly because northern Ontario's fairly stagnant economically. In general, almost any Canadian city with a strong economy and a population greater than about 100,000 has seen serious immigration in the last generation, mainly from non-English speaking nations.

This is particularly true of the larger cities - Calgary, where I live, is nearly as polyglot across the board as Toronto is, and its northeast neighbourhoods are like a pan-Asian diorama or something - but there are also unexpected little pockets nationwide. The ranchland hub of Brooks, Alberta, for example, is now per capita the most Sudanese municipality in Canada.
posted by gompa at 9:41 AM on December 4, 2007

Also, it occurs to me to mention that because historically there have been little French-speaking pockets nationwide and bilingualism has been official policy and on-the-ground fact for a good long time, maybe Canadians are a little more receptive to the idea of dealing with multiple languages and more skilled at accounting for them in our schooling than our southern neighbours.
posted by gompa at 9:43 AM on December 4, 2007

Here are some interesting maps of languages spoken in the US and Canada. Decide for yourselves.

Many posters seem to be referring to the fact that the Canadian population has more diverse language backgrounds because of immigration policies etc. They also emphasize the resources put into ESL. In Quebec, if the law hasn't changed since I last looked in, immigrants have to go to French schools. The net result: people speak one or both of the two national languages. That would tend to create the "homogeneity" within communities I was referring to.

Would someone from Finland please speak up???
posted by cogneuro at 9:48 AM on December 4, 2007

That would tend to create the "homogeneity" within communities I was referring to.

I don't quite follow your reasoning, cogneuro. Do you mean that recent Canadian immigrants are more likely than recent American immigrants to speak English? Or if like 75 percent of a high school student body in Northeast Calgary speaks a language other than English at home but all those kids also speak and write fluent English, that counts as homogeneity?

(And those maps? I don't know what they're measuring, but it definitely isn't mother tongue, or else the Canadian big-city parts would look like Jackson Pollack paintings.)
posted by gompa at 10:00 AM on December 4, 2007

I guess what I'm thinking about is the extent to which there are relatively linguistically-unassimilated groups in the two countries. (I'm not a sociologist; I don't know.) So, for example, we have a large Hispanic population and there is considerable controversy about their language experiences, the extent to which they do or do not eventually learn English and function outside Spanish speaking enclaves--and the extent to which our benighted bilingual education policies actually discourage learing English (by emphasizing teaching in the child's native, Spanish language). This is another thread.

I'm afraid this term "homogeneity" is subject to different interpretations and it doesn't capture the Canadian situation very well. I back off on that. But, yes, I did have in mind that native language (which may not be English or French) is not as good a measure of homogeneity as the degree of success in learning and functioning in one of the national languages.

The language issues are interesting (to me and others perhaps) but was only one part of my original post. We have more individuals at the lower end of the achievement distribution. Canada (and other countries) may well put more resources into helping those individuals. Voila, success.
posted by cogneuro at 10:22 AM on December 4, 2007

I wonder whether basic homework makes a big difference. In Poland, which is above average in reading and average in math despite bad teacher pay, kids come home with heaps of books and they go to work on them. They work through fat classic novels and lots of math problems with little dependence on technology other than paper and pencils.
posted by pracowity at 10:33 AM on December 4, 2007

And they cheat like crazy, pracowity. I would take any Polish results with a massive grain of salt. As a former teacher in a Polish HS, I do not recall students "going to work" on piles of books. I have distinct memories of desperate copying of homework in the halls, during the breaks between classes.

Part of the educational tradition is that for final exams, mothers sew little hidden pockets on the inside of their children's clothes, where the children hide elaborately designed and carefully folded microtexts covering the entire year's curriculum.
posted by Meatbomb at 11:08 AM on December 4, 2007 [3 favorites has favorites]

Variables ST11Q01-ST11Q03 (In what country were you and your parents born? 1-Country of test,2-Other country) and ST12Q01 (What language do you speak at home most of the time? 1-Language of test 2-Other national language 3-Other language) can give more than anecdotes. US and Canada have quite similar profiles there, except in Canada the handicap from secondary language group is smaller than almost anywhere. Finland is bilingual, where 5,5% minority speaks Swedish but almost all of them Finnish too. They probably did their test in Swedish-speaking schools in Swedish, so that kind of bilingualism doesn't show much in scores.

Estonians have 30% Russian speaking minority, without official status for russian language. Only 2% anyway claim to speak other than language of test at home, so probably there was russian option for the test available. They're fifth. Chinese Taipei, fourth, has 22% using their other language at home. Hong Kong, second, has again almost everyone doing test with their home language, but there half of the parents and 1/4th of students are born in other countries, and claiming Hong Kong to be culturally homogenous would be quite a stretch anyway.

Also, there are comparable homogenous cultures nearby, Norway, Iceland and Sweden in lesser amounts, but they didn't do well. Netherlands did better, for example.

Doing test in language other than what you speak at home or being first generation immigrant gives huge penalty (30-90 points) in european countries, about half of that in U.S. and Canada. Even with that penalty, immigrants in top countries pass native-borne from mediocre countries.
posted by Free word order! at 11:08 AM on December 4, 2007

Here are some interesting maps of languages spoken in the US and Canada. Decide for yourselves.

Those maps are very odd- they seem to indicate that there are large pockets of Iroquois speakers dotting the Bay Area, along with what appears to be New Yorker English. Or Midlands. Or perhaps its Bawlmerese English. Anyway, seems to leave out most of the languages that are actually spoken here, and is strangely color coded to boot.
posted by oneirodynia at 11:26 AM on December 4, 2007

Why is this all about Canada & the US?

Australia's results, all round, are quite reasonable. Out of 57 participating countries, we achieved the following rankings:

Scientific literacy: equal fourth (behind Finland, HK-China & Canada)

Mathematical literacy: equal ninth (behind Taipei, Finland, HK-China, Korea, Netherlands, Switzerland, Canada & Macau)

Reading literacy: equal sixth (behind Korea, Finland, HK-China, Canada & New Zealand)

To me, that suggests fourth place out of the fifty-seven overall, with the ranking being Finland, HK-China, Canada, then Australia.
posted by UbuRoivas at 2:52 PM on December 4, 2007

why is Canada so successful given this diversity?

My wildly speculative theory is down to ideology. In the US and UK there is more of a expectation and tolerance of educational failure. Eggs are cracked to make omelets. The focus is on making the innate best better and letting innate failures fail. Canada seems to have a more modest goal of ensuring equality of opportunity and training all students rather than revealing innate greatness of a few.

So the UK and US let many many students fall through the cracks while funneling resources to a select few. With those few there are great results. With the rest no so great. In Canada there isn't really a select few.
posted by srboisvert at 2:54 PM on December 4, 2007

Australia's results, all round... suggests fourth place out of the fifty-seven overall

And when you combine swimming and crocodile hunting, you guys go straight to the top of the list. Damn you Australia!

Do they mention the sample size for each country? Seems like it could be easy to skew just by giving the test to the right group of students - they certainly didn't test every 15 year-old in the US (did they?).
posted by GuyZero at 3:01 PM on December 4, 2007

I guess what I'm thinking about is the extent to which there are relatively linguistically-unassimilated groups in the two countries.

Linguistic assimilation is one of the cornerstones of Canadian identity and history, back to the plains of Abraham. It's on par with the American Revoution and the settlement of the west in US History. Heck, it goes back to the Hundred Years' War, except we don't have the convienent little channel between the French and the English.

Government spending on ESL/FSL is the best way to keep it a two-party system.

Someone should do a FPP on reasonable accomodation and le Commission de consultation sur les pratiques d’accommodement reliées aux différences culturelles.
posted by GuyZero at 3:07 PM on December 4, 2007

And when you combine swimming and crocodile hunting, you guys go straight to the top of the list. Damn you Australia!

Well, that's a bit unfair on the other countries, I think. As a matter of self-defence, swimming here necessarily entails crocodile hunting.

Or shark hunting, depending on where you choose to swim.
posted by UbuRoivas at 3:19 PM on December 4, 2007

The maps aren't that weird. It looks like the mapmaker punted on characterizing languages in urban areas. The languages are broken down by dialect. The colors are hard to tell apart although clearer in the regional blowups. Not clear where the data came from. No data about bilingualism etc. But, it looks like most people in America speak blue English. People in Canada speak French or English. There.

Of course there are many factors combining to yield these outcomes: relative economic, ethnic, and linguistic homogeneity seems important. We clearly have a greater spread of incomes and opportunities in this country, though.
posted by cogneuro at 6:22 PM on December 4, 2007

damn, that was supposed to have "ethnic and linguistic homogeneiy" crossed out!
posted by cogneuro at 6:24 PM on December 4, 2007

My Edmonton high school was just as ethnically diverse as any street in Toronto. You don't have to live in one of the big three (Montreal/Toronto/Vancouver) to get the same effects.

I think there are so many things that the US is doing wrong that no discussion here can encompass them all.

Anyway, why are we talking about them? Go Finland!
posted by blacklite at 2:58 AM on December 5, 2007