Author: Charles Taylor Kerchner

harles Taylor Kerchner is Research Professor in the School of Educational Studies at Claremont Graduate University, and a specialist in educational organizations, educational policy, and teachers unions. In 2008, he and his colleagues completed a four-year study of education reform of the Los Angeles Unified School District. The results of that research can be found in The Transformation of Great American School Districts and in Learning from L.A.: Institutional Change in American Public Education, published by Harvard Education Press. He writes the blog Mindworkers.com,

Analysis of LA Times series shows pitfalls of using test scores to evaluate teachers

Nearly half the rankings handed out to L.A. Unified teachers by the Los Angeles Times may be wrong. This is one of the conclusions reached by Derek Briggs and Ben Domingue of the University of Colorado at Boulder, who conducted a reanalysis of the data used by the Times in their value-added analysis of teacher performance. Using very strong language for the semi-polite world of social science, they concluded that the newspaper’s teacher effectiveness ratings were “based on unreliable and invalid research.”

At issue here is the validity of the original research conducted by Richard Buddin, the wisdom and responsibility of the Times in publishing individual teacher names, and the newspaper’s response to the reanalysis. In a policy environment where value-added analysis is actively being considered as part of how teachers should be evaluated, the reanalysis results are a big deal.

The teacher rankings story was published as an exposé last August under the subhead “a Times analysis, using data largely ignored by LAUSD, looks at which educators help students learn, and which hold them back.” In the Briggs and Domingue reanalysis, the which sometimes got switched:

More than half of the teachers (53.6%) had a different effectiveness rating in reading in the Briggs/Domingue alternative analysis than they did in Buddin’s original analysis. In math, about 40 percent of the teachers would have been in a different effectiveness ranking.

When extended to teachers who were labeled “ineffective” or “effective,” over 8 percent of those the Times identified as ineffective in teaching reading were identified as effective by the alternative analysis. Some 12.6% of the teachers graded as effective in the Times were found ineffective in the alternative analysis.

Why these differences? The answer is simple: not all value-added analyses are the same, and some are a lot better than others. The whole idea of valued-added assessment is to statistically control for the elements of a student’s experience that are out of the teacher’s direct control. It’s obvious that a student’s achievement at the end of the 5^th grade, for example, can’t all be attributed to that year’s teacher. Some of it results from the student’s family, some from prior teachers, and some is commonly attributed to economic and social circumstance. There are different ways of doing this, and researchers who work with value-added analysis usually approach this task with a great deal of transparency because they know that the outcome depends on the method they use.

Here’s what we know about the two analyses: First, it makes a big difference who gets counted. Briggs/Domingue got different results when they analyzed the Times data using the same technique used by Buddin, who works at the RAND Corporation but did the value-added analysis on his own time. As it turns out, each of the researchers had made different decisions about which teachers and which students to exclude from the study. Almost all studies exclude cases because data on some students are incomplete or the teacher may have had only a few students with valid test scores. The rules about who is counted and who is not turn out to be very influential to a teacher’s ranking.

Second, it makes a big difference what outside-the-classroom factors are included. Buddin’s technique included relatively few of these factors, fewer than used in the leading studies in the field. In his model, a teacher’s value-added score was based on student California Standardized Test performance moderated by a student’s test performance the prior year, English language proficiency, eligibility for federal Title I (low income) services, and whether he or she began school in LAUSD before or after kindergarten.

Briggs, who is the chair of the research and methodology program at the University of Colorado education school, and Domingue, a doctoral student, added a handful of new variables. Particularly, they looked at what I call the “tracking effect,” whether a student came from a high performing or low performing classroom or school. The results led to starkly different conclusions about rankings for nearly half the teachers.

They also found that when the standard statistical confidence level was put in place, between 43% and 52% of the teachers could not be distinguished from the category of “average” effectiveness. As a consequence, the technique would be useless in ranking the broad swath of teachers in the middle for bonuses or salary bumps.

Briggs/Domingue also disputed Buddin’s assertion that traditional teacher qualifications have no effect on effectiveness rankings. In their analysis, experience and training count.

If the differences in results were just a disparity of opinion among statisticians, the question would be resolved in academic journals, not the media. But reporters Jason Song and Jason Felch, and the Times editorial board, have raised the use of value-added assessment to a matter of public debate. Secretary of Education Arne Duncan has endorsed the idea, and other newspapers are following the Times’ lead in demanding teacher-identified student test data from their school districts.

So, how the public policy process deals with criticism of value-added techniques is important. On Monday, Felch’s story about the statistical reanalysis appeared under a headline that said, “Separate study confirms many Los Angeles Times findings on teacher effectiveness.” It does not, and I do not understand how one could reasonably draw such a conclusion from the highly readable 32-page research report written by Briggs/Domingue. I invite you to read their report, a summary of it, and their response to Monday’s story. The only large point on which the two analyses agree is that there is big variation between the most and least effective teachers in Los Angeles Unified.

Is that a surprise to anyone? The Times’ contribution has been in raising the issue of variations in teacher effectiveness politically, and, as I said last August, the original Song-Felch story was one of those rare “this changes everything” moments in journalism. But whatever contribution raising the issue of variable teacher performance made is undermined by the continued defense of the technique’s application against a barrage of criticism from scholars and statisticians. The reanalysis grounds this criticism in the original data. Those who want to use value-added assessment to pay teachers or fire them need to acknowledge the technical difficulties involved.

The Times also raised the question of sponsorship of the Briggs/Domingue study by the National Center for Education Policy, a reliably left-leaning think tank that has financial ties to teacher unions. Briggs was quick to claim his independence, saying that his work had received no interference at all from the Center: “I have no ideological axe to grind at all about the use of value-added assessment or the L.A Times. I do feel passionately about the quality of the research.”

The substantive underlying question is whether value-added analysis can help improve schools or teaching in any practical way.

My response is that it can, but only if we moderate our language. As the final paragraphs of the Briggs/Domingue analysis note, we need to avoid two extreme positions:

Unless value-added analyses can be shown to be perfect they should not be used at all. Value-added measurements do not need to be perfect to be a great improvement over the techniques currently used in teacher evaluation;
Any critique of the method or its conclusion constitutes an endorsement of the status quo. There are many useful improvements to current data use practice. Some don’t use value-added analysis at all, and others use it in more sophisticated and less mechanical ways than the Times analysis.

We all need to be realistic about what the findings show and don’t show. Just because we have the statistical chops to perform an analysis, that doesn’t mean that it’s good or that it should rapidly be married to a pay and personnel system to “winnow away ineffective teachers while rewarding the effective ones.” Causal analysis may be the Holy Grail of statistics, but casual application of it is not.

At the very least, the reanalysis underscores the invalidity of extending causal analysis to individual teachers, and the folly of releasing teacher names and their rankings.

Charles Taylor Kerchner is Research Professor in the School of Educational Studies at Claremont Graduate University, and a specialist in educational organizations, educational policy, and teachers unions. In 2008, he and his colleagues completed a four-year study of education reform of the Los Angeles Unified School District. The results of that research can be found in The Transformation of Great American School Districts and in Learning from L.A.: Institutional Change in American Public Education, published by Harvard Education Press.

Imagine teachers as free agents before adopting pay for performance

U.S. Education Secretary Arne Duncan and a raft of supporters in the foundation world fervently want to replace the tried-and-true teacher salary schedule with pay-for-performance schemes. They should be careful what they wish for.

The idea seems straightforward: Replace raises conditioned on years of service and education beyond the minimum required for a teaching license with conspicuous rewards for good teaching measured, at least in part, by student test scores. Unsuccessful teachers would see the lack of salary advancement as a sign that they should seek another line of work. Successful teachers would be incentivized, take home tangible rewards, and encourage other teachers to follow in their wake. A circle of virtue would ensue.

Hardly anyone would disagree that high-performing teachers deserve rewards and recognition, but the policy mavens seem not to anticipate that departing from the single salary schedule might cost a lot of money and change teaching in unanticipated ways. It’s about Talent. Not talent, as in singing or dancing well, but Talent as a commodity that can be bought or sold.

The switch from considering teachers as economically interchangeable widgets to scarce and valuable Talent would have profound effects, increase the overall wage bill for education, and give teachers unions their greatest organizing opportunity in half a century. Talent, once identified, will demand to be paid.

The cautionary tale about rewarding performance comes from Malcolm Gladwell. In a New Yorker article published in October 2010, the always-provocative Gladwell tells the story of Talent’s rise to economic royalty starting in 1966, when Marvin Miller took over the Baseball Players Association. Talking to the San Francisco Giants player Bobby Bonds, Miller said, “If we can get rid of the system as we know it, then Bobby Bonds’s son … will make more in one year than Bobby will make in his whole career.” In fact, in 2005 Bobby’s little boy Barry was paid $22 million, more than his father and all his teammates made in their entire careers.

Miller’s insight was that labor organizing could be applied to valuable and scarce abilities known collectively as Talent, and that Talent could make huge demands on Capital. Labor relations was no longer to be a contest between Capital and interchangeable worker Labor, but Capital’s capitulation to highly valuable Talent. Goodbye reserve clause. Welcome to the world of seven- and eight-figure sports salaries, billion dollar bonuses for investment bankers, and outsized salaries for executives.

It is hard to imagine such galloping salaries in schools where teachers are economically treated as interchangeable parts. But before Marvin Miller, it was hard to imagine baseball players as other than big kids under the paternalistic control of club owners. The history of categories suggests that once a category is created, economic value follows. One of the most recent examples is in the creation of Board Certified Teachers, which was followed shortly thereafter by salary differentials for them. We should expect the same from Talented Teachers, those with proven track records of lifting student achievement. Indeed, this is what Secretary Duncan and others desire.

But what if the Talent of these teachers was recognized and promoted by their schools? Colleges and universities already do this and offer large salary premiums to their stars. Increasingly, they also offer insecure part-time employment to those who teach many of their core undergraduate courses. Some charter school organizations are built around conspicuous displays of Talent. Some traditional school districts also poach Talent from other places with promises of favorable salary scale placement and other work life enhancements: a lab of technological goodies or a schedule with plenty of time to teach the things Talent wants to teach. If these things can exist in the current teacher-as-widget salary schedule, imagine what might happen if Talent recognized its worth and conspicuously organized around its protection and advancement?

Imagine teachers as free agents, teachers with agents, teachers unions as agents. Imagine school districts demanding salary caps, and lawsuits demanding that salaries be corralled into something like a civil service salary schedule so that schools in less wealthy areas could compete. Imagine the division of teaching into conspicuous rewards for Talent and slim pickings for the rest. Imagine brutal competition among Talent.

And maybe Talent drags up the salaries of everyone. Between 1985 and 2007 the average baseball player salary increased tenfold, not counting inflation. Over the same years the average teacher salary increased less than 2.4 times, from $20,694 to $50,478, barely keeping up with inflation, according to the Digest of Education Statistics.

One suspects that the advocates of conspicuously rewarding Talent haven’t reckoned the wages that Talent demands.

The piece also appeared in Conditions of Education in California, the blog of the PACE, Policy Analysis for California Education.

Jerry Brown’s ed plan reflects realism toward school reform

Gov.-elect Jerry Brown’s education plan is smart, pragmatic, and, in one respect, a little bit pregnant.

Brown’s adviser and Stanford professor Mike Kirst suggested I reexamine the governor-elect’s plan, this with the assurance that the governor-elect means to do what he said. A close reread opens the eyes to possibilities for substantial reform that can be accomplished, or at least begun, in these tough budgetary times. Smart. Pragmatic.

(My comments follow with page and paragraph references to Brown’s plan for those who want to follow along with the original.)

In a world where “you campaign in poetry, but govern in prose,” Brown didn’t get the poetry part, but the prose is good. He and Kirst have moved well beyond the simplistic I’ll-be-an-education-governor, blow-it-all-up rhetoric. When was the last time anyone heard a candidate say that schools in California are making progress toward measured cognitive achievement goals? (page 1, para 2). In fact, they are. By no measure are the schools where they need to be, but they are moving in the right direction. A couple weeks ago I talked with Laura Schwalm, the veteran superintendent of Garden Grove, whose district won the Broad Prize honoring urban districts several years ago. Even from a relatively high level of achievement, Schwalm says, “our worst school is now doing better than our best school was then.” There are similar stories throughout the state.

And when ever has a governor-elect said: “I approach this task with some humility, and a realization there is no silver bullet that will fix everything. Education improvement takes time, persistence, and a systematic approach.” Humility? Refreshing (page 1, para 3). Brown comes to this humility in part because he’s actually tried to turn around schools. He started two charters in Oakland, and like every other businessperson, philanthropist, or freelance politician who has tried to fix a school, he comes away with the conclusion that it is very hard work (page 2, para 5).

Brown would start with higher education: updating the master plan that in the 1960s created the state’s three-tier system of colleges and universities (page 3). It’s a good place to begin. Higher education plays a different role in California’s society and economy than it did a half century ago, and the state’s plan needs to reflect current reality. Some kind of post-secondary education is virtually a necessity for a young person to get the kind of job that will allow them to raise a family, buy a house, and pay enough taxes so the next generation will have these opportunities, too. But higher education rests on the shoulders of the public schools; they need to work as one system.

The pathway from elementary and high school through higher education needs to be level and well lighted. The new master plan needs to create that pathway, because it is the systemic key to many of the rest of the elements in Brown’s plan, including improving the high school graduation rate (page 7). Many of the needed efficiency gains can come from better transitions from schools to colleges and universities. Fewer surprises yield fewer remedial courses. Alignment of community college and university standards (page 4, para 1) is a good starting place. The effort needs to extend to the high schools, where university requirements drive the curriculum, but where graduates still experience mismatches between what courses they took and what is accepted at state colleges and universities.

Much of the science-technology-engineering-math reinvigoration, which Brown wants to see, starts deep in elementary schools, when 9-year-old girls learn that they really like math, and that’s okay, and that there are futures for them in science and technology. If they don’t get 5^th grade math, they won’t line up in the engineering school admissions line. Likewise, much of the reinvigoration of vocational education that the governor-elect seeks requires a technical education pathway that tells students that combining “head and hand” is real school, not a dumping ground.

Part of creating a level, well-lighted pathway requires overhauling the state’s testing program, as Brown suggests (page 4). The tests California students take largely don’t help them or the school systems get smarter. Scores and analyses arrive too late and in forms that are hard to use. The tests are good for naming and shaming of poor-performing schools and districts, but not much else. Given the $100 million we spend each year on them — and that figure doesn’t include the huge hunks of school time spent on test prep — we aren’t getting a very good deal.

English Language Learners instruction is a big uneven flagstone in the pathway to success. There are almost 1.5 million students, about 23 percent of the total elementary and secondary enrollment, who do not yet speak English well. The needed overhaul extends beyond the adoption of new instructional materials and leveraging of federal funds mentioned by the governor-elect (page 7). The ESL testing program is a hurdle, not a help. The monetary incentives are backward; they reward schools for keeping students in English-learner status, and ESL instruction is often not well integrated into the general course of study.

A good portion of the Brown education plan is aimed at redressing a historic drift of authority toward Sacramento. Since the passage of Proposition 13 in 1978, the state has become education’s paymaster, and the state has channeled money by creating scores of targeted programs, called categoricals. Brown wants to drastically prune them and deliver more program and fiscal control to local school districts (page 4). At the same time, he wants to rewrite and simplify the 12-volume school code. Brown also seeks to redress another historic drift, one of narrowing the curriculum to fit the mandates of the current testing system. He encourages a “broader vision of what constitutes an educated person” (page 6). Good ideas, but difficult to execute; each barnacle on the hull of the school statutes grew there because of a well-intended legislator or interest group. Usually, only states that were forced by the courts, such as Kentucky, are successful in a wholesale overhaul. And broadening the curriculum to areas not tested swims against the federal government’s current. Still, moving authority downward has a populist ring and can be implemented without increased expenses.

Brown also seeks some other changes, such as improving the attractiveness of teaching to high-ranking college students (page 5) and increasing the number of magnet schools (page 7). Charters get mild encouragement,(page 7). Taken together, the elements of the plan are amazingly pragmatic. Brown knows there’s no money to buy change. He and Kirst have found the places where change is possible without large infusions of cash. but not support for massive expansion

A missing piece: technology

Yet, the part of the plan that struck me as having the most possibility was only mentioned in a couple paragraphs, leaving the ideas underdeveloped, a little bit pregnant. Young people in California are changing the way they read, write, communicate and learn. But in the state whose education system gave birth to the information-processing revolution, very little of its progeny have changed the structures of schooling.

The governor-elect wrote about higher education, he advocated exploring online learning and new technologies “to the fullest,” to expand access, increase productivity and reduce costs (page 4). He also advocated expanding online and virtual capacity in science, technology, engineering and math (page 7). These would be important initiatives. But they only begin to capture the sea change that is taking place in learning and teaching and its capacity to shape California education at all levels. Schools, colleges, and universities are rapidly adopting online courses and experimenting with new ways of interacting with students. California foundations, such as Hewlett, have been in the forefront of creating an open-source movement that is making college and university courses and materials freely available. The trend of technology is to break down the century-old pattern of batch-processed learning and open the door to instruction that is more student-friendly and available on demand. Technology adoption will intensify, even if nothing is done in education policy.

However, as governor, Brown has an opportunity to invest in a learning infrastructure that connects kids, teachers, and parents, that allows every student in the state access to high-quality learning material and first-rate teaching. Unlike the current CALPADS design, which if it works will be good at collecting data on students, a learning infrastructure would help them learn and manage their own learning. A learning technology system, such as the one the Scottish government has developed, would have six elements.

First, it would provide information to students and parents. It could begin with report cards and state test data, and should rapidly expand to more fine-grained data. For example, English language learners and their parent should have access to the progress students are making toward fluency and the benchmarks they still need to complete. Second, an information infrastructure would connect families and teachers through a secure communication link such as that now used for patient-physician communication at Kaiser Permanente. Third, the system would provide direct assistance to students through online tutoring. Fourth, it would begin, at least, to open source the curriculum, and to make it improvable by teachers from throughout the state. Fifth, it would channel or link to the rapidly developing capacity for direct instruction.

Finally, it would open the possibility of online testing. It would provide self-paced examinations and certification of competency in ways that break down the relationship between time spent in classrooms and progress toward graduation from high school. Only when this relationship — one of the most enduring aspects of an education system designed early in the last century — is broken can we begin to expect substantial productivity gains in public education. An external examination system tied to student progress also creates a system in which both decentralization and standards-based accountability are possible.

If the governor had one big bet, one big investment to make, it should be in this already developing embryo.

Charles Taylor Kerchner is Research Professor in the School of Educational Studies at Claremont Graduate University, and a specialist in educational organizations, educational policy, and teachers unions. In 2008, he and his colleagues completed a four-year study of education reform of the Los Angeles Unified School District. The results of that research can be found in The Transformation of Great American School Districts and in Learning from L.A.: Institutional Change in American Public Education, published by Harvard Education Press.

Negotiate student achievement goals into teachers’ contracts

Collective bargaining lends itself to lots of different conversations, but it’s hard to talk about what matters most: how spending money will or won’t make education better.

An illustration: Jacob Adams, a Claremont Graduate University colleague, has edited a new book, Smart Money, which shows that effective school districts spend their money differently than those with lower student achievement gains. In the book, and in an Education Week commentary, he argues for a “simple, powerful principle,” linking collective bargaining and other forms of resource allocation to student achievement. The smart-money approach is to adopt cycles of continuous improvement — goal setting, instruction, assessment and analysis — to tailor resource allocation to classroom needs. But where, in a school system filled with large and small interests, including pensions and pocketbook issues, are these conversations to be held?

School districts spend their money when they negotiate their labor contracts; sometimes more than 80 percent of the current operating budget goes to pay employees’ wages and benefits. Negotiators for management and the unions spend months in sometimes tense conversations over how and what people are to be paid and under what rules they will work. Except in a small number of “reform union” districts, student achievement is hardly mentioned during these negotiations, and this is not an accident.

When California teachers gained the right to bargain collectively in 1976, the framers of the statute followed the National Labor Relations Act industrial assumptions about how authority and responsibility are to be divided. School managers determine the outcomes they want to see, and they create work processes and technologies in the form of a required curriculum and instructional practices. They assess the results through inspection of teaching and by testing students. Teachers are expected to faithfully carry out the prescribed routines and be obedient to orders. Insubordination is dealt with more harshly than incompetence.

Even under the narrowest possible construction of the scope of bargaining, “wages, hours, and conditions of employment,” have a huge patterning effect on how schools use their resources. Union leaders frequently assert that they know what resources count. Small class sizes are frequently mentioned. Yet, nowhere in collective bargaining are union and management required to talk about how all the money they are allocating will change student achievement.

I believe they should be. In an American Journal of Education article, Julia Koppich, who also contributed to Smart Money, and I argued in favor of a small but radical change in labor law: Make student achievement goals a mandatory subject of agreement. Without student achievement goals that both labor and management sign on to, no one gets a raise, no one gets a contract at all. (Full text available here.)

This modest idea modifies the function of collective bargaining. Labor-management negotiations have been legally structured to accommodate what unions (mostly) want to talk about. Some topics are mandatory in the sense that if one side raises them they must be bargained in good faith, and whatever dispute resolution mechanisms the law provides must be used. Some topics are permitted, allowable if both labor and management want to discuss them. (Some topics are also forbidden.) Long treatises and much case law have been devoted to sorting out what has to be negotiated from what may be. But nowhere does the law specify that the parties have to come to agreement on a particular topic. For example, if the union neglected to raise the issue, school districts would not be required to discuss the length of the school day or year.

It may be that neither labor nor management wants to discuss student achievement goals, but I believe it is in the public interest to require them to do so. No one would think of negotiating a contract with a building contractor without specifying the outcomes and the processes to be used; why should teachers be different? For teachers and school managers, specifying goals strengthens accountability to them. Specifying goals requires talking through which objectives are more important than others. Putting achievement goals on the same table as wages, hours, and working conditions requires that labor and management face the reality that how money is spent has a relationship to achievement, and that personal preference and private interest may have to bend in the achievement of the school’s core purpose.

It is reasonable to raise the objection that schools and teachers have lost the ability to set their own goals. Particularly since the passage of the California and federal accountability statutes and the implementation of statewide testing, explicit goals are being externally imposed on school districts. But even if the collective bargaining agreement simply incorporated these externally generated goals, talking about them would have the advantage of determining how resources would be used to meet those goals. There are virtually no other arenas in the school budgeting process that bring money and intent together with such force.

If teachers and school managers were able to bargain goals, they would probably reach for things other than test scores. Both teachers unions and school administrators chafe under test-score accountability. More authentic assessments that teachers value, such as portfolios of student work, are often pushed aside. Negotiated goals might include changes in graduation rates, numbers of students taking a college or career-ready curriculum, or increased numbers of second-language learners becoming classified as fluent in English.

At the end of negotiations, the contract would be required to contain student achievement goals. These could be ambitious or ridiculously low. Since the contract is a public document, teachers and management would take the heat for setting the bar below where the public thinks it should be. Without a student goals clause, there would be no labor contract. If a contract were to be signed without student achievement goals, it could be challenged in court or before the public employment relations board. Any citizen could bring such a challenge.

Negotiating student achievement goals would gradually change labor doctrine and practice while leaving all the familiar structures in place. The negotiated contract would continue to be the centerpiece of the labor relations. Unions would continue to talk about economic issues and worker protections. The grievance procedure would continue in force. The one change would be that the parties would be required to talk about student achievement and to bring to those discussions their often unvoiced ideas about what causes learning in schools.

Charles Taylor Kerchner is Research Professor in the School of Educational Studies at Claremont Graduate University, and a specialist in educational organizations, educational policy, and teachers unions. In 2008, he and his colleagues completed a four-year study of education reform of the Los Angeles Unified School District. The results of that research can be found in “The Transformation of Great American School Districts” and in “Learning from L.A.: Institutional Change in American Public Education,” published by Harvard Education Press.

‘Waiting for Superman’s’ half-truths and heroes can move you to tears

Waiting for Superman, the Davis Guggenheim documentary about public education, is headed for the theaters with more hype and about as much substance as a B-grade Western.

As in the B-grade Western there are villains, heroines, simplistic truths, and a pull at your heartstrings.

The plot line of Superman follows five children and their families from New York, Los Angeles, Washington, D.C., and Silicon Valley as they — like thousands every year — go through school choice lottery programs, hoping to get into the school they want. There’s joy and tears, more of the latter.

The movie does what it intends: It emotionally involves the viewers in the struggle of children and their families to find schools that work for them and to avoid some of the most troubled public schools, aptly named “dropout factories.”

The villains and heroes are easily identified. People who represent teacher unions, like Randi Weingarten, the president of the American Federation of Teachers, are villains. People who represent charter schools and those who want to kick ass and take names in school districts are heroes and heroines. Michelle Rhee, the chancellor of the D.C. schools, is a heroine. Geoffery Canada, the charismatic founder of the Harlem Children’s Zone, is a hero and gives the film its title. Canada recalls that, as a boy, he cried when his mother told him that Superman did not exist, “because no one was coming with enough power to save us.”

The search for a Superman has been an enduring feature of the school reform movement, going back at least a quarter-century: If there would only be a strong, charismatic leader or reading or math program, our problems would be solved. Like many charismatics before him, Guggenheim thinks that he’s found the answer. “In recent years, we’ve cracked the code. The high-performing charter schools, like KIPP and others, have figured out the system that works for kids in even the toughest neighborhoods,” he told Dom Giordano of the Philadelphia Daily News.

In other words: charter schools good, unions bad. Unfortunately, this is a convenient half-truth.

There are some very good charter schools, and there are charter school leaders who are very good at promoting their schools. But virtually every study of charter school performance comes back with the same message: The results are mixed. One of the largest, most careful studies. done at Stanford, found that only 17 percent of charter schools had statistically significant higher results in math, while 37 percent of the charter schools had worse results than corresponding public schools.

The more we learn about charter schools, the more clearly we understand that the difficulty of bringing promising practices to scale is as complex in those schools as it is in traditional public schools. As Ted Kolderie, who influenced the founding of charters, writes, “No one would ask whether eating at home is better than eating out: Clearly it depends on what you eat in the restaurant and what you eat at home . … A charter is simply permission to start a school: No student learns anything from a charter.”

And, yes, there are some very bad labor-management practices. Both national teacher unions have been slow to adopt a strong quality agenda, and to represent teaching as an occupation as well as teachers as employees. The irony of the film is that it casts Weingarten in the villain’s role, when she has pushed the quality agenda forward, often with substantial internal opposition.

Clearly, unions and management need to negotiate better ways of evaluating teachers and removing those who can’t or won’t teach. Clearly, unions and management need to find ways that seniority is used to place teachers; if new innovative schools are to flourish, they need to be able to pick teachers whose beliefs and practices match those of the school. Clearly, the system needs to provide much greater variety in the types of schooling available, the same specialization that attracts parents by the thousands to wait in line for admission or enter the lottery processes depicted in the film.

These changes need to happen now, and perhaps the best thing about Superman is the sense of urgency it brings to the public policy discussion. But Superman stands in a long line of would-be action heroes that have declared a crisis in public education, and despite its hubris it is far from the best in its analysis or prescription. It’s correct at the end, though. There is no Superman; there’s only you.

Charles Taylor Kerchner is Research Professor in the School Educational Studies at Claremont Graduate University, and a specialist in educational organizations, educational policy, and teacher unions. In 2008, he and his colleagues completed a four-year study of education reform of the Los Angeles Unified School District. The results of that research can be found in The Transformation of Great American School Districts and in Learning from L.A.: Institutional Change in American Public Education, published by Harvard Education Press.