Grad rates trending up – or down

If California’s graduation and dropout rates were a pair of jeans, they’d have an “irregular” sticker on them. Sure, social science research has a reputation for squishy results, but it’s still a bit jarring when a renowned researcher describes certain data as “bogus.” Although he said it with an ironic laugh, that was the first word that popped into Russell Rumberger’s mind when I asked him about the California high school graduation rates in “Diplomas Count,” an annual report from Education Week.

Rumberger is an education professor in the Gevirtz Graduate School of Education at UC Santa Barbara, where he founded the California Dropout Research Project and has been conducting research on school dropouts for 25 years. His new book on the subject, Dropping out: Why students drop out of high school and what can be done about it, is due out later this year.

A sharp dip followed by a sharper rise

Where California stands nationwide in graduation rates according to Diplomas Count (click to enlarge)
Where California stands nationwide in graduation rates according to Diplomas Count (click to enlarge)

The statistics in question are in the California supplement of the report (available for purchase from EdWeek here), showing changes in graduation rates between 1998 and 2008. For much of the time they’re fairly stable, from a few tenths of a point to a little over one percent from year to year. Over the course of the decade, California’s graduation rate increased from 67.5 percent to 73 percent, staying close to the national average. But in between there’s a dramatic shakeup and recovery.

It started in 2005, when the state’s graduation rate was 70.1 percent. Within two years, by 2007, it had fallen to 62.7 percent, the sharpest decline in the ten-year period. A year later, in 2008, it surged back up by more than ten points to 73 percent. “Pretty wild,” said Rumberger. “The bottom line is all these numbers are estimates, and estimates have errors.”

California runs its own numbers, and they’re quite different. Actually, California runs two sets of numbers,

Diploma Counts shows a sharp drop in Calif graduation rate not seen in other analyses (Click to enlarge. Courtesy Bob Nichols, SVEF)
Diploma Counts shows a sharp drop in Calif graduation rate not seen in other analyses (Click to enlarge. Courtesy Bob Nichols, SVEF)

but more on that in a moment. The official statistics that the state sends to the U.S. Department of Education, for No Child Left Behind reporting, show a downward trend between 1998 and 2008, falling from a high of 87 percent to about 80 percent. But even at their lowest point, those graduation rates are still higher across the board than EdWeek’s figures.

And just to complicate it a little more, that second set of numbers that California prepares has graduation rates heading up after a 2006 downturn, but coming in lower than EdWeek. So we have three sets of calculations, all using data from the same source, obtaining different results and different trends.

It’s all in the formula

It’s no better nationally. Rumberger recently served on a committee of the National Research Council and National Academy of Education that examined the various measurements that states use to determine their graduation and dropout rates. The committee found “widespread disagreement” about the best measurements and their uses. Looking at the year 2005, the committee found three different school completion rates published by the U.S. Department of Education, Editorial Projects in Education (a project of EdWeek), and the Annie E. Casey Foundation. Some researchers suspect the problem is with the way the rates are calculated and not with the numbers used to make those calculations.

In its final report, the panel found most formulas to be flawed and occasionally skewed by politics. “At a time when policy makers are vitally interested in tracking the incidence of dropping out of school, they are faced with choosing among substantially discrepant estimates that would lead them to different conclusions regarding both the size of the drop out problem and how it has changed in recent years,” they wrote.

The most common calculations are aggregate counts, the cumulative promotion index (CPI), and cohorts. Aggregate is the simplest and bluntest, comparing the number of graduates to the number of ninth graders four years earlier. The CPI is based on how many students progress from grade 9 to 10, 10 to 11, 11 to 12, and then ultimately graduate. Karl Scheff, with the California Department of Education, says CPI is better than aggregate but still doesn’t get at what happens to individual students. That’s the true cohort measure and it’s the brass ring of data.

Calling on CALPADS

California has been collecting this data for five years, ever since assigning individual student identifiers as part of the CALPADS student data system.  Up until now, even though districts have been sending in student-level data, the state has still been aggregating it.  “We just added them up and plugged them into the traditional calculation,” said Scheff.  This year should be the first time they have a full cohort of students from grade 9 through graduation, but now funding for CALPADS is up in the air.

Gov. Brown proposed cutting the budget for CALPADS in his May revise.  Both the Assembly and Senate have restored the money but Scheff says it’s not clear whether the Governor will veto it.  With CALPADS, state officials will have a robust data source that can track students who move out of

"The actual rate is somewhere in the middle," says Russell Rumberger of the California Dropout Research Project (click to enlarge)
"The actual rate is somewhere in the middle," says Russell Rumberger of the California Dropout Research Project (click to enlarge)

state, transfer to a private school, or spend a fifth year in high school in order to provide a much more precise look at graduation and dropout rates.

Until then, we’ll have to sort through two, three or four reports each with its own interpretation of the data.  Rumberger tries to be as precise as possible when giving presentations.  “I’ll show the two state rates,” he said, “and what I tell people is the actual rate is somewhere in the middle.”

Additional Resources:

A More Accurate Measure of California’s Dropout Rate, June 2010, California Dropout Research Project.

Independent Evaluation of the California High School Exit Examination: 2010 Evaluation Report, Volume 1, Oct. 2010, Human Resources Research Organization

Public School Graduates and Dropouts From the Common Core of Data: School Year 2008-2009, National Center for Education Statistics

Now 43rd in per-student spending

Praised for its high academic standards and accountability measures, dinged for its low scores on the national standardized tests, low high school graduation rates, and disadvantages of high rates of poverty and non-English speaking households, California fell squarely in a crowded, mediocre middle – 30th among the states, in this year’s annual ratings by Education Week’s Quality Counts. It earned a C grade: 76.2. The perennially top-ranked states are Maryland (tops with B+, 87),  New York (84.7) and Massachusetts (82.6). Twenty states were between California and 10th West Virginia at 79.9.

The publication’s per-pupil spending ranking is the figure most commonly cited in California’s debate over school spending, because it’s adjusted for regional costs. This year, California ranked 43rd among the states and Washington, D.C.; last year it was 46th.   The $8,852 spent per pupil in 2008 – before the full impact of the recession hit California’s schools – was $2,371 below the national average of $11, 223. It will probably be headed lower once 2009 and 2010 figures are out. California is  squeezed between #42 Washington, just ahead of Arizona, and a freefall behind top-spending, low-cost Wyoming’s adjusted figure of $17,114.

California spends 3.5 percent on K-12 schools as a percentage of state taxable resources, 10 percent below the national average; it ranks 36th.

Quality Counts uses 50 measures, equally weighted. Thirty-six states joined California with a C.

The measures fall into six categories:

Chance for Success: Grade C (national average C+), rank 42nd . This measurement looks at indicators from preschool to adulthood. California ranked 8th in the nation with 80 percent of  kids in kindergarten, 17th with 50 percent in preschool, and 24th with 54 percent of adults enrolled in post-secondary institutions or holding a degree.

At the same time,  California ranks last in the nation, with only 61 percent of parents fluent in English (national average is 80 percent).

State Achievement Indicators: Grade D- (national average D+), rank 46th.  This category is largely based on problematic comparisons of state scores on the National Assessment of Educational Progress, in which California’s fourth and eighth graders have consistently ranked among the lowest in the nation. One problem is that NAEP doesn’t test algebra, which most California eighth graders take.

California does rank  8th in the percentage of high school students passing Advanced Placement tests (25 percent), and that percentage has increased impressively over the past decade.

Education Alignment Policies: Grade B- (national average C+), rank 14th. California does well because the state defines school readiness, has kindergarten standards aligned with first grade, and aligns high school career tech courses with industry expectations and post-secondary ed requirements.

School Finance: Grade C (national average C), rank 24th. Despite its low per-pupil spending, California ranked squarely in the middle overall. Its finance system, based on state taxes and not local revenues, is more equitably distributed than most states, for all its complexities and anomalies (high-spending districts in the Bay Area that are not dependent on state aid).

Standards, Assessments, Accountability: Grade A- (national average B), rank 17th. Aligned standards are California’s strength, and its accountability measures, sanctioning low-performing schools for low API scores, get a nod from Ed Week. However, its tests are multiple choice and not scaled from one grade to the next.

Twenty-seven states have benchmark assessments to guide teachers and students; California does not. Two dozen states have extended response questions that measure deeper levels of understanding than multiple choice questions; California is not one of them. (The Brown administration has made rethinking assessments a priority.)

Efforts to Improve Teaching: Grade C (national average C), rank 20th. This one mystifies, since California has scaled back teacher training dramatically and eliminated state incentives for teachers and principals to work in low-performing schools (some districts offer incentives), and unlike 32 states, has no internship program for aspiring principals. It has a high student-teacher ratio and no requirements for professional development for teachers. Unlike 15 other states, it does not require annual evaluations of teachers; 13 states tie evaluations to student achievement, California does not.

California does pay its teachers as much as comparable professions (I’ve seen conflicting data). The average teacher salary of $68,093 is second highest in the nation, next to New York, according to the National Education Association.

Impact of the recession

This year, Ed Week examined how states responded to the recession with changes in policies.

California is one of 35 states that have cut spending for K-12. It is also one of :

  • 21 states that  have allowed greater flexibility to districts over spending decisions;
  • 10 states that allowed a short school year or shorter school day or week;
  • 11 states that loosened regulations on class size.

But other states have acted where California has not. It is not one of:

  • 15 states that have changed tenure rules in response to the economy or other factors;
  • 22 states that have  changed pension benefits or funding.

It also is not the one state (Arizona) that changed layoff criteria. California is one of only 11 states that require teacher layoffs based on seniority.

State lags in new math index

California has, by far, the largest percentage of eighth graders taking algebra. But that’s about all it can crow about in Education Week’s first Math Progress Index, which was published last week.

By most measures – scores on the “nation’s report card” (National Assessment of Educational Progress), improvement on those scores over the last six years,  closing the achievement gap in math, and hiring experienced math teachers – California is far behind the most successful states, and often behind the national average. Continue reading “State lags in new math index”