It started with a video of Brown published by Slate. In it she said “[t]wo out of three eighth graders in this country cannot read or do math at grade level. We are not preparing our kids for what the future holds.”
Tom Loveless from the Brookings Institute (followed by Carol Burris and Diane Ravitch) took to Twitter to challenge the accuracy of Brown’s statement, and they feverishly demanded she correct it. Brown declined. Things got hyper-technical from there. The point of conflict centered on the precise meaning of “grade level,” and the way “proficiency” is measured by the National Assessment of Educational Progress (NAEP).
Brown clarified her position in an email to the Washington Post, saying “If I were trying to be completely and utterly precise then I would have specified “grade-level proficiency,” instead of “grade level” in the context of NAEP scores. But any reasonable person or parent can rightly assume that if their child is not reading at grade level, then their child is not proficient.”
That answer was not sound in Burris’ ears. She felt Brown’s equating student “grade level” performance with NAEP’s “proficiency” was “not merely a matter of semantics.”
NAEP tests produce a continuum of scores, much like the SAT. In order to make scores more understandable to the public, the four categories (Below Basic, Basic, Proficient and Advanced), each aligned with performance descriptors, were created and assigned to four ranges of scores by the NAEP governing board (NAGB). In doing so, the NAGB made it clear that “Proficient is not synonymous with grade-level performance.”
Loveless, who has written extensively about NAEP, said the following in his email correspondence with me:
“The cut point on NAEP is much too high [to be considered grade level].
On the first claim, that proficiency isn’t the same as grade-level performance, it’s true that NAEP governors admit that.
On the second claim, that NAEP’s proficiency criteria is too high, NAEP’s governing board say their standard for “proficiency” is high, but with a purpose [for an easy to read explainer on NAEP, click here]:
NAEP’s standards indicate what students should know and be able to do according to the National Assessment Governing Board and are generally considered aspirational. Standards may differ for different tests because they have different purposes. States may construct their standards for the same purpose as NAEP, or they may use them for defining minimum competency, determining promotion to the next grade, or addressing other purposes. Under NAEP’s legislation, NAEP’s achievement levels remain in a ‘trial’ status, to be used and interpreted with caution, until the Commissioner of Education Statistics (on the basis of a congressionally mandated, independent evaluation of NAEP) determines that NAEP’s achievement levels are reasonable, valid, and informative to the public.
That said, we don’t have a common standard for grade-level performance. States set their standards all over the map. Ravitch once described that problem as “50 states, 50 standards, 50 tests,” then concluded “the evidence is growing that this approach has not improved student achievement. Americans must recognize that we need national standards, national tests and a national curriculum.”
She’s changed her tune on that recommendation, but the conditions and research supporting her previous support for national standards have not changed.
We still don’t have national tests that illuminate through comparison the performance of students across state lines. That’s probably why people – like Brown and others – look to NAEP, the so-called Nation’s report card, for a common marker of achievement.
Given to students in every state, the NAEP’s “proficiency” reflects a common measurement of students’ “competency over challenging subject matter, including subject-matter knowledge, application of such knowledge to real-world situations, and analytical skills appropriate to the subject matter.”
Brown isn’t the first (and won’t be the last) to use NAEP proficiency as a proxy for grade-level performance because it’s the closest thing we have.
This past January presidential candidate Marco Rubio wrote “Two-thirds of our kids can’t read at grade level” in an article posted to Medium.
His statement was fact checked by Politifact, and on the basis of input from several experts they concluded “Rubio said that “two-thirds of our kids can’t read at grade-level.” The numbers from the NAEP reading test for fourth and eighth graders provide some support for his statement. However, experts said that there is no accepted definition of what being at grade-level” means under the NAEP tests. While Rubio’s use of NAEP‘s higher “proficient” standard is a reasonable choice, experts said, it’s not the only one…”
Basically, the fight between Burris and Brown over the “two-thirds” statement is a draw.
A 2014 NPR story about NAEP results also conflated “proficiency” and “grade-level,” saying “[i]n the NAEP test, achievement is broken down into three levels: basic, proficient and advanced. “Basic” indicates partial mastery of the subject, “proficient” is grade-level performance, and “advanced” indicates superior work.”
Just this past April an Education Week article said “Results for NAEP, known as the Nation’s Report Card, are also reported at three achievement levels: basic, proficient, and advanced. “Proficient” indicates students are successful with challenging, grade-level content.”
Consider something Ravitch said in 2010. An NPR articles reported “[s]ome states contend that 80 to 90 percent of their children are proficient readers and have math proficiency as well, Ravitch notes. But in the same states, only 25 to 30 of the children test at a proficient level on national tests such as the National Assessment of Educational Progress.”
That sounds a hair away from saying, as Brown and Rubio did, that two thirds of our students can’t read or do math at grade level.
If true, that is the real problem, and it’s one that Brown seems eager to address, if only she can get through the logical sludge that flows so prolifically from Burris, Ravitch, and their key amplifier at the Washington Post, Valerie Strauss.
I have more to say about this, look for second post soon.