IN THIS ISSUE

This special issue of AFT's Closer Look addresses the continuing debate surrounding the 2003 NAEP test scores for charter schools. The results show that students in other public schools generally outperform charter school students in reading and math.

Setting the Record Straight: Low NAEP Scores for Charter School Students
Some charter school advocates refused to acknowledge charter schools' mediocre performance on NAEP tests. Their arguments were imaginative but not based on the facts. More . . .  

False Claim #1: NAEP scores are inappropriate for evaluating charter schools.
Yet they're the "gold standard" for other public schools? More . . .

False Claim #2: The NAEP results are tainted because they come from the AFT.
The recently released scores are not AFT data. The AFT, whose former president Al Shanker first proposed charter schools, produced the report that should have been released months ago by federal officials. More . . .

False Claim #3: The administration released all the NAEP charter school results long ago.
Officials continue to suppress important data collected with the NAEP assessments. Their handling of these data violates longstanding precedents, written policies and common sense. More . . .

False Claim #4: The "snapshot" nature of NAEP results is unfair to charter schools.
Some attackers of the NAEP results claim, without evidence, that established charter schools show more progress than other public schools. NCES could settle the issue by releasing information it has collected. Researchers shed some light on this issue recently, and the picture for North Carolina's charter schools wasn't pretty. More . . .

False Claim #5: NAEP data fail to account for the charter schools' hard-to-teach kids.
AFT researchers made "apples to apples" comparisons based on poverty, central city location and race. (Are the critics claiming that other public schools have few disadvantaged kids?) More . . .

False Claim #6: New charter schools shouldn't be evaluated.
It's difficult to understand why people who concede that students fare poorly in new charter schools would advocate sending more and more children to such schools. More . . .

False Claim #7: Charter schools close when students don't learn.
Charter schools have been shut down for mismanagement, financial problems and other reasons, but when was the last time a charter school closed because students weren't learning? More . . .

* * *

SETTING THE RECORD STRAIGHT: LOW NAEP SCORES FOR CHARTER SCHOOL STUDENTS

Last week's disclosure by AFT researchers of the 2003 test scores for charter school students generated a great deal of attention. The scores, from the National Assessment for Educational Progress (NAEP), were buried on a U.S. Department of Education Web site. The Department failed even to announce that the results were on the Web site, which is available only to those with the knowledge, inclination and research skills to access them.

The data, reported on page one of the New York Times, show charter school students generally do worse than students in public schools, even when geography and poverty are considered. African-American students in charter schools scored lower than African-American students in other public schools, but the difference was not statistically significant. AFT researchers presented the data with a minimum of interpretation and scrupulously followed the usual format of NAEP reports.

The New York Times subsequently editorialized that "the data casts doubt on a central provision of the No Child Left Behind Act (NCLB) that encourages the states to hand over failing schools to commercial companies and nonprofit community groups that want to run them as charter schools." Other news reports and commentary also saw the data as a warning against the rapid proliferation of charter schools. Scripps-Howard columnist Dan K. Thomasson wrote, "Until charters meet well-thought-out and proven standards of education . . . the government should go slow in supporting them." A Daytona Beach News editorial concluded, "The data suggest what many already know: It's time to slow down the political passion to create more charter schools without better standards to measure their overall performance." In the New York Times, columnist Samuel Freedman wrote discerningly (and perhaps optimistically) that "the most perceptive supporters of charter schools do recognize the AFT report for the wake-up call that it is."

Other editorials questioned the federal government's decision not to publicize the data. An editorial in The State, a South Carolina newspaper, posited that "if the U.S. Department of Education had revealed more plainly what it knew about charter school performance last year, the numbers probably wouldn't have been the Page One news they were." And, indeed, the federal commissioner of education statistics told the Times, "I guess that was poor publicity on our part."

Not surprisingly, a small group of public school bashers went on the attack, using a variety of media outlets and communications tools (and hastily assembled talking points) to launch an attack on the message and the messenger--firing away at the AFT, the Times and, by extension, the NAEP tests, which have often been described as "the nation's report card." 

The assault on the NAEP data is particularly ironic coming from people who say they share the AFT's belief in research-based reforms, high standards and accountability. AFT's uncovering of the data shows that charter school students, on average, are not meeting high standards. (A recent posting on the National School Boards Association's blog pointed out the hypocrisy of much of the criticism and outlined other inconsistencies in the arguments.)

Charter schools' original bargain included greater freedom and greater accountability. But some charter school advocates are making excuses. During George W. Bush's first campaign for the presidency, he spoke often of how "the soft bigotry of low expectations" harmed poor and minority students. Some charter school supporters now seem to be applying that same "soft bigotry" not only to disadvantaged students but also to charter schools. By doing so, they are lowering the bar for charter schools and manufacturing a permanent excuse for charter schools with low-achieving students.

What follows is a critical look at the most common false claims made by those wishing to dismiss the poor NAEP results for charter schools.

False Claim #1: NAEP scores are inappropriate for evaluating charter schools.

According to the U.S. Department of Education, NAEP is "the only nationally representative and continuing assessment of what America's students know and can do in various subject areas." And when the U.S. Department of Education releases NAEP scores, it refers to the results as "the nation's report card."

NAEP tests are so highly regarded that the No Child Left Behind Act, signed by President Bush, requires states to participate in NAEP as a condition of receiving federal Title I funds. Peggy Carr, the National Center for Education Statistics' associate commissioner for assessment, has called NAEP scores "the gold standard for monitoring the academic progress of America's children," a view widely shared by education scholars.

False Claim #2: The NAEP results are tainted because they come from the AFT.

The recently released scores are not AFT data. They're a presentation by the AFT of the NAEP data. The AFT presentation used a format that the NCES and the National Assessment Governing Board (NAGB) regularly use to report the scores. The National Center for Education Statistics has not challenged the NAEP results as reported by the AFT.

Even the attacks on the data show that the attackers respect the NAEP results. Chester E. Finn Jr., an education department official during the Reagan years, wrote that AFT's release of the NAEP results was "chock-full of union spin and political agenda." But the AFT followed the NAEP format and released the data with only the minimal interpretation that accompanies all NAEP data.

"The data is probably what it is," said NAGB chairman Darvin Winick. "NAEP is pretty accurate; there shouldn't be any question about the results." And Finn's own responses to the data--he called the scores "dismayingly low," "a wake-up call" and "sobering"--show that he believes the scores are an accurate reflection of charter schools' performance.

Some critics also have misrepresented AFT's position on charter schools. In 1988, the late Al Shanker, who served as president of the AFT, introduced the idea of charter schools. Shanker envisioned experimental public schools with the potential to improve education for all students, but, even in the earliest years of the charter school movement, he was concerned about the its direction. A 2002 AFT report, one of many studies that suggest Shanker was right to be concerned, found that charter schools, by and large, have not lived up to their promise and need greater accountability. With too many charter schools taking shortcuts to avoid the high standards that AFT advocates, it's not surprising to see mediocre results like those on the 2003 NAEP tests.

The AFT, despite what Finn has said, is not a charter-hating union. Rather, its leaders and members insist on reforms that work--in charter schools and in other public schools.

False Claim #3: The administration released all the NAEP charter school results long ago.

This argument--that the data already were accessible--ignores several critical facts.

First, NCES is still sitting on data related to the charter school NAEP scores. A special questionnaire was administered to the charter school sample, and that questionnaire has information that will help researchers further analyze the charter school data. However, the answers to the questionnaire have not been released by NCES and are not, as of this writing, on the NAEP Data Tool.

Second, NCES never announced publicly that the NAEP scores were available on the NAEP Data Tool--an unprecedented and unwise handling of data that were supposedly made public. Independent researchers could learn of the data's existence on the Data Tool only by reading the minutes of NAGB board meetings. But, until AFT's researchers uncovered the charter school scores, no outsider seemed to know the data could be accessed.

Third, although NCES has announced it will release its report on the NAEP scores at the end of this year, it also plans to release a report with statistical adjustments and predicted outcomes. Not only is this unprecedented in NAEP's 35-year history, it also violates a 1989 NAGB resolution prohibiting officially reporting NAEP with "adjusted" or "predicted" results because they "would be subject to serious methodological and political challenges. . . ." As then-NAGB member and charter school advocate Chester E. Finn Jr. said in 1994 when NAGB reaffirmed its 1989 board resolution, "while it was proper for researchers to prepare adjusted scores, it would be wrong for them to [be] part of a government report, such as NAEP; . . . such scores would damage the credibility of [the] program." AFT welcomes such analyses, which in the past have been conducted only by independent researchers.

False Claim #4: The "snapshot" nature of NAEP results is unfair to charter schools.

AFT's presentation of the NAEP scores, as noted above, is consistent with past presentations of NAEP scores for other public schools and private schools, which also provide a one-year snapshot. Predictably, few charter school zealots have yet to complain about the quality of the research when those NAEP snapshots have shown that students in other public schools are not performing well.

But this request for more than a snapshot may not result in a pretty picture of charter school performance. Some of the NAEP critics called for studies that measured the growth of charter school students over time. A comprehensive study of North Carolina's charter school performance, released days after the national NAEP data were reported, did just that. And the results don't reflect well on charter schools.

Helen Ladd, one of the study's authors, said, "Our study finds that charter school students perform less well on average in charter schools than they would have in other public schools and the negative effects of attending a charter school are large." She concludes, "The bottom line is that North Carolina's system of close to 100 charter schools appears to have reduced overall academic achievement in the state."

If, as some of the critics argue, NAEP tells us little or nothing because it is not a measure of student progress, then adequate yearly progress (AYP)--the accountability formula in NCLB--also tells us little or nothing. Despite having "progress" in its name, the formula doesn't measure student progress or gains. AFT, which has examined the problems with AYP, would welcome an honest and comprehensive discussion of the strengths and weaknesses of NAEP as well. That discussion, however, would need to address more than NAEP's limitations as it applies to charter schools, which enroll less than 2 percent of American schoolchildren. An honest discussion also would have to consider NAEP's effectiveness in measuring performance at private schools and other public schools that enroll more than 50 million students.

False Claim #5: NAEP data fail to account for the charter schools' hard-to-teach kids.

The AFT made every fair comparison possible with the information available on the NAEP Data Tool. The NAEP data, as reported by the AFT, showed that economically disadvantaged students--identified here through their eligibility for free or reduced-price lunch--fared worse in charter schools than did similarly disadvantaged students in other public schools. AFT also compared inner-city charter schools to inner-city public schools and found, once again, that charter school students performed worse than other public school students in math. Both findings were statistically significant.

The NAEP data also revealed that African-American students in charter schools scored lower in math and reading than African-American students in other public schools, but not at statistically significant levels. The NAEP attackers, desperate to counter data that reflect poorly on charter schools, crowed about this lack of statistical significance in one area, pretending that it demonstrates that charter schools are superior to other public schools. In fact, it suggests that charter schools have failed to raise achievement for minority students at a faster rate than public schools have.

William Howell, Paul Peterson and Martin West--researchers committed to the proliferation of charter schools--write in the Wall Street Journal that the low NAEP scores are caused by the "poor, minority children" in charter schools, pretending that this is a special condition of charter schools. Of course, it's also a description of many public schools in every major city in the United States. Charter schools in some districts are less diverse than other public schools, and some charter schools attract elite, high-performing students. In their Wall Street Journal editorial, the pro-charter, pro-voucher trio claims--with no evidence--that charter schools are more disadvantaged than other public school students. But in states where researchers have examined student populations, it appears that charter schools generally:

• Isolate students by race and ethnicity;
• Enroll fewer special education students than comparable local school districts;
• Enroll fewer students with the most serious disabilities; and
• Enroll fewer English language learners than comparable local school districts.
  
  
  

False Claim #6: New charter schools shouldn't be evaluated.

It's difficult to understand why people who concede that students fare poorly in new charter schools would advocate sending more and more children to such schools. Yet, critics like Howell, Peterson and West seem to believe that newness is an excuse for charter schools to be unaccountable. They write, "Almost one-third of the charter schools nationwide were less than two years old when the NAEP was administered, raising doubts about whether even a sophisticated analysis of NAEP data would be relevant once charter schools have had time to become well established." But if charter school proponents truly wanted to hold charter schools to high standards, they wouldn't insist on increasing the number of children attending ineffective schools that may become good schools in five or 10 years.

Researchers could tackle the question of whether the newness of charter schools is to blame for the low NAEP scores, but only if NCES releases the responses to the charter school questionnaire, which would allow researchers to identify the age of charter schools in the NAEP sample. Such research has been done at the state level. A recent study of charter schools in North Carolina indicates that long-established charter schools are not performing well. The study, which provides just the type of analysis called for by Howell, Peterson and West, evaluating long-established charter schools, finds that "the negative effects of charter schools in North Carolina remain statistically significant and large even for schools that have been operating for five years."

False Claim #7: Test scores don't matter because charters close if kids don't learn.

The ideology of some charter school advocates leads them to propose a "failure equals success" argument based on the theory that a charter school will be shut down if it fails to help students learn. Thus, they argue, charter schools should be immune from accountability measures that are based on low test scores. But more pragmatic supporters of charter schools, realizing that such closures rarely happen in the real world, will take action to improve the mediocre NAEP scores for charter school students and ensure that charter schools meet their part of the bargain.

If you have a comment or suggestion for AFT's Closer Look, let us know.

AFT's Closer Look is a publication of the public affairs department of the American Federation of Teachers, 555 New Jersey Ave. N.W., Washington, DC 20001, 202/879-4458.