Sometimes our sound-byte-attention-span, this-way-or-that-way culture can yield some quiet amusement. Researchers in California did what looks to me, on a casual skim, like a pretty decent study. Of course it’s correlational, and they’re careful to point out that correlation isn’t causation. They also note some of the limitations inherent in a demographically restricted sample from one state. I find their preliminary report is skimpy on the statistical basis of their conclusions (a single table doesn’t give much detail about a roughly 400 item survey given to 5,500 people at 257 schools), but given how long its been since I really worked over a regression analysis, I should be grateful.

The study, “Similar Students, Different Results: Why Do Some Schools Do Better?” found that some things schools could do (for example, having a coherent curriculum) improved the school’s result on California’s academic performance index, which seems to mean, on a statewide battery of mostly multiple choice questions. (It appears also – hold on to your hat – that teaching to the test improves achievement on the test. Did I say teaching to the test? I’m sorry, the correct phrase is “classroom instruction guided by state academic standards.")

The Washington Post reported on the brouhaha that resulted from one of the study’s conclusions:
Parents’ Effect on Achievement Shaky
Other Factors May Play Greater Role, Study Says.

The article reports “But a new study of low-income public schools in California has concluded that several other factors, including teaching the state’s rigorous academic content and getting experienced teachers, have much more influence on achievement than does parents’ involvement.” Somehow, the “more influence” conclusion has led to “a national debate on the subject, with some parents … saying the study is correct and others saying parental influence should not be so quickly dismissed.” (Did you hear “dismissed” in the study’s conclusions?)

Ironically, the motto of EdSource, who sponsored the study, is “Clarifying Complex Education Issues.” The authors of the study leapt to defend themselves: “The study did not find that parent involvement is not important or not related to student achievement,” but within their sample, some other factors showed more connection to performance on the standardized tests.

Listening carefully to the authors’ protestations, the Post article interviews other experts who assure us that “Building positive relationships … is vital.” “…parent involvement is a key factor in the achievement gap and in improving low achievement.” “Schools should make unequivocal public commitments to involving parents” “there is too much research showing parents playing a significant role to ignore them.”

[Deep sigh]

• When a 23 page initial report gets reduced to a couple sentences, you may lose something.
• Any study needs to be evaluated in terms of its specific purpose, population, and methodology. Its conclusions aren’t general, but specific to the questions asked and how evidence was turned into answers.
• Science builds, assuming it does build, on a slow accretion of such specifics.
• When “The Media” get their hands on your research, run for cover!
• Oversimplifications mislead. Except for the previous sentence.

As Paul says, “Further bulletins as events warrant. Film at 11:00.”