Yeah, I think I made up that word, disserves. But I mean it; this new plan to use NAEP cut scores to grade schools and students on our state test would be a tremendous disservice.
As many of you may know, Wisconsin was granted its waiver from the federal No Child Left Behind law a couple of weeks ago. As part of earning that waiver, the Wisconsin state Department of Instruction had to promise to be eternally more vigilant about student achievement.
There are two very visible differences between the pre-waiver and the post-waiver world in Wisconsin. One is that the state’s test is changing. Wisconsin has signed on to the nearly-national Common Core State Standards, and is part of a consortium of states developing a test aimed toward those standards. This will change in the 2013-14 school year.
The other visible change happens now, and it is, as I suggested above, changing the cut scores.
If you don’t know what that means, the inestimable Alan Borsuk laid it out here; the equally inestimable Erin Richards digests and interprets the DPI memo (pdf) that explains just what a difference the new cut scores mean:
Only 35.8% of Wisconsin’s WKCE test-takers in third through eighth and 10th grade in fall 2011 scored proficient or better in reading, and just 48.1% scored proficient or better in math. Compare that with March, when the state released 2011 WKCE results that showed 78% and 82% of students scored proficient or better in math and reading.
As I said: Devastating. My dad would have killed me if I’d been anything south of proficient. Starting today, a whole giant slice of Wisconsin’s students suddenly are.
The reason is simple: NAEP standards (NAEP is the National Assessment of Educational Progress, colloquially called the nation’s report card) are higher. No, scratch that; NAEP standards are different.
Wisconsin’s old system used four levels of achievement, from low to high: Minimal, Basic, Proficient, and Advanced. NAEP also uses four levels of achievement, from low to high: Below Basic, Basic, Proficient, and Advanced.
Hey! you may be thinking, Those match up nicely! Three of the achievement levels are the same!
Except they’re not. Whereas Wisconsin’s old definitions are here–get them fast before they’re retconned out of existence! Proficient in Wisconsin sensibly used to mean that a student “[d]emonstrates competency in the academic knowledge and skills tested on WKCE for that grade level.” That seems like a C (on that scale of A-F) to me, and that’s always the way Wisconsin has thought of it–students scoring as proficient are those who have not necessarily excelled, but who have learned what they needed to learn.
However, NAEP doesn’t treat “proficient” in the same way. I’ll let Diane Ravitch, who used to sit on the NAEP board and knows what she’s talking about, explain NAEP’s definitions:
Advanced is truly superb performance, which is like getting an A+. […] Proficient is akin to a solid A. […] Basic is akin to a B or C level performance. Good but not good enough. […] And below basic is where we really need to worry.
So NAEP’s proficient is like Wisconsin’s old Advanced, where a student “[d]emonstrates in-depth understanding of academic knowledge and skills tested on WKCE for that grade level.” An A+, if you will.
In other words, the NAEP scores simply don’t align with Wisconsin’s old scores: NAEP’s basic is actually Wisconsin’s proficient.
Now, okay, there is, I dunno, debate about whether Wisconsin’s old standards were too low. The afore-mentioned Alan Borsuk beat this drum pretty regularly when his beat was regular — here’s one example. However, others have plotted Wisconsin against NAEP and found our standards fairly middle-of-the-road — here’s one example, if on the slightly low side of the middle. I am not here to have that debate today.
However, I think it’s ridiculous and simply beyond any kind of belief that a state that consistently ranks in the top or above average for, for example, ACT scores and graduation rates and — we learned this week — college completion has only a third of its students reading at a “proficient” level. To point to our successful, and even our doin’ pretty good, students and say they are not proficient when they by any reasonable definition are is just downright cruel.
And doing so doesn’t add “accountability” so much as it adds stigma.
Part of what’s happening here is that this country is on an insane spiral to define proficiency up. The late and lamented (so very lamented) Gerald Bracey wrote about exactly this:
I have repeatedly observed that the NAEP results do not mesh with those from international comparisons. In the 1995 Trends in International Mathematics and Science Study, or TIMSS, assessment, American 4th graders finished third among 26 participating nations in science, but the NAEP science results from the same year stated that only 31 percent of them were proficient or better. […]
Because we have scores for American students on NAEP and TIMSS and scores for students in other countries on TIMSS, it is possible to estimate the performance of other nations if their students took NAEP assessments.
How many of the 45 countries in TIMSS have a majority of their students proficient in reading? Zero, said Phillips. Sweden, the highest scoring nation, would show about one-third of its students proficient while the United States had 31 percent. In science, only two nations would have a majority of their students labeled proficient or better while six countries would cross that threshold in mathematics.
Bracey quotes a study done in 1991 when NAEP first started applying labels: “[T]hese standards and the results obtained from them should under no circumstances be used as a baseline or benchmark … the procedures used in the exercise should under no circumstances be used as a model.” And yet here we are, applying them to Wisconsin’s students in a giant disservice to their actual achievement.
i evaluate to yes even if there's no image 
Jay,
I really enjoyed the post! You raise some excellent points about not stigmatizing the labels that are assigned as the result of standardized assessment. I was intrigued the following paragraph:
“However, I think it’s ridiculous and simply beyond any kind of belief that a state that consistently ranks in the top or above average for, for example, ACT scores and graduation rates and — we learned this week — college completion has only a third of its students reading at a “proficient” level. To point to our successful, and even our doin’ pretty good, students and say they are not proficient when they by any reasonable definition are is just downright cruel.”
True, Wisconsin’s average composite ACT score is comparatively high to other states. However, using these scores to discredit low K-8 proficiency on the KCE ignores a major problem in our state: In the WI 2011 graduating class, 78% (37,011) of the students who factored into their ACT average were white. Only 3,528 students (7%) in this average were African American and 2,397 (5%) were Latino. (http://www.act.org/newsroom/data/2011/pdf/profile/Wisconsin.pdf) The main point here is that our high state ACT average is NOT a valid representation of what all students in our state can do. The WKCE is given to all students. This is why there’s such a disparity in student performance on these two standardized measures.
The fact that so few non-white students are taking the ACT is a major problem, and this is a great reason to avoid citing our state ACT average as an indication of quality education in our state.
Thanks for the post and I’m looking forward to reading more in the future!
-Mark
@markketter
All measures are in some way imperfect, but consider this about the ACT: Every junior in the Milwaukee Public Schools takes the test–college bound or not, white or black or Hispanic, rich or poor. Even so, even with Wisconsin’s worst students included, our kids overall do pretty well.
Yes, Wisconsin is a whiter-than-average state and that may account for some of our above-averageness (Alan Borsuk makes that argument). But it simply defies common sense to pretend or claim that 2/3 of them don’t read at grade level. Please.
So we have the same teachers, same schools, same spending, same curriculum, same kids. Last year the kids were OK. This year the kids suck.
I guess this is supposed to build a fire under someone…presumably the teachers.
It should be obvious that the push to corporate led Charter schools is behind the recent led criticism. Charter schools allow less standardize testing and can pick students to whom publicly funded education money that comes out of public school budgets is given.
This is the prize for which tens of millions of out-of-state dollars is being spent on Republican candidates.
Actually, no. Charter schools are public schools and must offer all the same standardized testing and follow most of the same rules–including teacher licensure and special-ed non-discrimination–that public districts do.