In clean­ing out my open web­pages, I came across the video above in an impor­tant post at Wired blogs, and it hard­ly mat­ters that the post is from last Octo­ber (yes, I keep too many tabs open in Fire­fox). Rhett Allain argues that there’s an inverse rela­tion­ship between how much stan­dard­ized test­ing stu­dents expe­ri­ence, and how much learn­ing they expe­ri­ence, and Ken Robin­son, in the video, argues that stan­dard­ized test­ing assumes that there is a sin­gle stan­dard way of learn­ing, or that such stan­dard­iza­tion is desir­able. The video touch­es on a wide range of points beyond that, and is well worth watch­ing, but this issue of stan­dards and stan­dard­ized tests is well worth pondering.

In my day job, I spend a lot of time think­ing about sci­ence stan­dards in the dif­fer­ent states, and about how they trans­late into stan­dard­ized tests, into text­books, and into class­room prac­tice. In terms of see­ing evo­lu­tion taught, and taught accu­rate­ly and hon­est­ly, the move­ment to estab­lish strong state edu­ca­tion stan­dards has been very good. From the mish­mash of the infa­mous “10,000 democ­ra­cies” (there are actu­al­ly about 17,000 large­ly autonomous school dis­tricts across the coun­try), these stan­dards give some degree of order. Hav­ing clear stan­dards gives text­book authors a clear guide, and as the cov­er­age of evo­lu­tion has increased in stan­dards, so too the cov­er­age of evo­lu­tion in text­books has improved. It’s hard to mea­sure what’s hap­pen­ing in each sci­ence class­room in each dis­trict, but bet­ter text­books and bet­ter stan­dards and bet­ter test cov­er­age of evo­lu­tion can all be expect­ed to trans­late to bet­ter evo­lu­tion edu­ca­tion, and that means bet­ter sci­ence education.

But the stan­dard­iza­tion trend also wor­ries me. If you estab­lish clear stan­dards, it only makes sense to have some sort of assess­ment to ensure that peo­ple are learn­ing what they’re sup­posed to be learn­ing. And once you have an assess­ment, it seems fool­ish not to insist that the assess­ment ought to car­ry some sort of con­se­quences. So you get the cur­rent mess of No Child Left Behind/Race To The Top, in which teach­ers are pun­ished for even mod­est year-to-year vari­a­tions (I recall Brad Delong run­ning the num­bers on one exam­ple, but can’t locate it now that I need it!), and class time is devot­ed to test prepa­ra­tion, not learn­ing. I wor­ry espe­cial­ly that in sci­ence class­es, stan­dard­ized test­ing and an empha­size on stan­dards-based edu­ca­tion could too-eas­i­ly result in a pro­found mise­d­u­ca­tion: teach­ing sci­ence-as-ency­clo­pe­dia (a list of things to mem­o­rize), rather than sci­ence-as-process (a way of test­ing claims that requires a par­tic­u­lar mind­set, cre­ativ­i­ty and insight, and also sub­ject knowledge).

There are sure­ly ways around that chal­lenge, tests involv­ing lab prac­ti­cals and oppor­tu­ni­ties to design exper­i­ments, and stan­dards that empha­size those aspects of sci­ence, and I remain hope­ful that some rea­son­able mid­dle ground can be found. Sci­ence is the same in every state, and in every coun­try. Hav­ing clear stan­dards makes sense, since stu­dents often move between dis­tricts with­in a state, and increas­ing­ly across state lines, and it’s help­ful for teach­ers to know what stu­dents are already sup­posed to know. And some sort of test­ing based on those stan­dards makes sense, because those stan­dards should rep­re­sent some­thing real about what we think stu­dents should know, and stan­dard­ized tests are an impor­tant tool for see­ing whether that impor­tant con­tent is actu­al­ly being learned.

The prob­lem comes when the tests take on a big­ger role. We go too far when stu­dent per­for­mance comes to dic­tate school fund­ing or teacher salary or reten­tion, or when the desire to do well on tests caus­es teach­ers to drop impor­tant lessons. Or, as Ken Robin­son observes in the video above, when the test­ing rein­forces harm­ful inequities in soci­ety, or rein­forces harm­ful men­tal habits and forces med­ical treat­ments on chil­dren with learn­ing styles and men­tal skills ill-suit­ed to a regime of stan­dard­ized test­ing (and the dull lec­tur­ing style such tests tend to force on teachers).

All of which reminds me of a clas­sic Cos­ma Shal­izi post from 2007, about the her­i­tabil­i­ty of intel­li­gence, and what sort of intel­li­gence we could plau­si­bly expect selec­tive breed­ing to increase. Cos­ma begins by not­ing:

Doing well on stan­dard­ized, mul­ti­ple-choice tests calls for cer­tain sorts of cog­ni­tive skills, cer­tain kinds of abstract prob­lem solv­ing. Maybe more exact­ly, you need both apti­tude at under­stand­ing explic­it rules for manip­u­lat­ing sym­bols, com­mu­ni­cat­ed to you through the medi­um of writ­ing, with very lit­tle con­tex­tu­al infor­ma­tion to help you make sense of the mes­sage, and you need to be will­ing to fol­low those rules, even when they are point­less. These are skills that come from mak­ing your way through an indus­tri­al, or more pre­cise­ly bureau­crat­ic and mass-lit­er­ate, soci­ety. (Shades of Luria!) These are skills you are more apt to learn if you grow up in a house­hold which is already high­ly lit­er­ate, etc., than if your par­ents and neigh­bors are all dis­placed peas­ants or harassed pro­les. It’s cer­tain­ly not sur­pris­ing if some­one who grows up in a house­hold of intel­lec­tu­als (that is, clerks) finds these habits easy to learn.

Q: So the anal­o­gy sug­gests that IQ scores are…?

A: A proxy for the skills and habits encour­aged by a bureau­crat­ic soci­ety; skills and habits which can be at once high­ly her­i­ta­ble (because of strong trans­mis­sion through fam­i­ly and neigh­bors) and high­ly learned (with­in the scope of what it is bio­log­i­cal­ly pos­si­ble for humans to learn and inter­nal­ize). Innate abil­i­ty need­n’t enter into it at all. The impli­ca­tions for democ­ra­cy would be near­ly nil.

And then extends the anal­o­gy:

Sup­pose that our new alien over­lords showed up tomor­row, and after demon­strat­ing that resis­tance is futile, decide to insti­tute a selec­tive breed­ing pro­gram. They tie every­one’s tubes just before puber­ty, and then at age 25 every­one is giv­en a test in which they must prove cer­tain the­o­rems about non-Abelian Yang-Mills field the­o­ries; those who pass are allowed to breed, those who fail are per­ma­nent­ly ster­il­ized. If this per­sist­ed for, say, a thou­sand years, I am quite con­fi­dent that a ran­dom­ly select­ed human being from 3007 will be much more like­ly to be able to prove those the­o­rems than a ran­dom­ly select­ed mem­ber of the present population.

Q: In this sce­nario, what­ev­er you call it –

A: How about “Clas­si­cal Lumps and Their Quan­tum Descen­dants”?

Q: I’m going to pre­tend you did­n’t say that. So, how would that come about?

A: How should I know? On the basis of the Bald­win effect, and the oth­er sorts of cou­pling between indi­vid­ual- and pop­u­la­tion-lev­el adap­ta­tion, I’d expect it to hap­pen some­how. It could be that peo­ple will evolve to have geo­met­ric intu­itions about non-com­mu­ta­tive Lie groups. Or they might get real­ly good at mem­o­riz­ing proof-length bits of text, if that kind of thing would get past the alien over­lords. Or — and some­how this sounds more plau­si­ble to me — they might just find study­ing real­ly inter­est­ing and enjoyable.

Q: And why does that last sound more plausible?

A: Because if you look at the actu­al sci­en­tif­ic lit­er­a­ture on exper­tise, the thing which stands out for impor­tance is mas­sive, delib­er­ate prac­tice. This takes inter­est.

…

If IQ real­ly cor­re­lates with the abil­i­ty to flour­ish in an indus­tri­al soci­ety (and I’m quite pre­pared to believe that), then it is, as I said last time, a mea­sure­ment of the abil­i­ty to nav­i­gate paper-push­ing bureau­cra­cies — to learn to manip­u­late arbi­trary abstract explic­it rules, and to do so on com­mand. Pre­sum­ing that peo­ple who don’t man­age to pull off at least some min­i­mum lev­el of this make very unat­trac­tive mat­ing part­ners, and so have below-aver­age repro­duc­tive suc­cess, then those of us in devel­oped coun­tries have spent the last one or two cen­turies breed­ing for docil­i­ty, in both sens­es of the word.

Q: You’re say­ing that –

A: We have met our alien over­lords, and they are us.

So many aspects of suc­cess in 21st cen­tu­ry Amer­i­ca rely on abil­i­ty to suc­ceed on stan­dard­ized tests, that this is cer­tain­ly not implau­si­ble. The more weight we place on these tests, the more we’ll rein­force the genet­ic cor­re­lates of stan­dard­ized test suc­cess, and the more we’ll select against (or med­icate away) diver­si­ty of learn­ing styles, of teach­ing styles, and men­tal habits — diver­si­ty which is not inher­ent­ly harm­ful but which does­n’t fit neat­ly into a bub­ble sheet.

So this is the ten­sion: I want to know what tests say, but I don’t want the tests.