How should we measure our schools if not by API?

Ever since California and the federal government placed the weight of a school’s success on standardized test scores with the Public Schools Accountability Act and No Child Left Behind, there’s been a backlash against overreliance on high-stakes testing.

The question of what else should be considered in rating schools is the topic of this week’s forum, “Yes, but….”

Our opinion and policy makers are Darrell Steinberg, President pro Tempore of the State Senate; David B. Cohen, a National Board-certified high school English teacher; education lobbyist and legal counsel Fred Jones; former California Superintendent of Public Instruction Bill Honig; and Jeff Camp, chair of the Education Circle of the Full Circle Fund philanthropy organization.

We hope you’ll keep the conversation going with other readers, and use the comment section to ask questions of this week’s contributors.

Darrell Steinberg: Reflect a well-rounded education

Darrell Steinberg
Darrell Steinberg

The Academic Performance Index has served a worthy purpose over the past 11 years, but let’s face it: It is, at best, an incomplete indicator of student achievement and school performance.

Gov. Brown’s veto of Senate Bill 547 left in place a measurement tool that sends one signal, and one signal only, to our schools: Get your standardized test scores up. At the elementary level, the API is almost exclusively focused on scores in just two subjects, English language arts and mathematics. At the middle and high school levels, no credit is given for keeping students on track to graduation.

Striving for the perception of steady improvement under this narrow accountability regime, many of our schools have responded with a laser focus on bubble tests. Such focus comes at the expense of a whole range of offerings that parents, the business community, and students themselves value: college and career preparation at the high school level; science, history, arts, and music across the grades; physical education; and opportunities for leadership and community engagement.

We need an accountability system that reflects the elements of a well-rounded education, and that connects public education to the needs of the 21st Century economy. I sought to begin that work by replacing the API with a new Education Quality Index, balancing test results with other important measures of school success. I have invited the Govenor to join me in crafting a new approach for next year. At minimum, it should contain the following elements:

  • Rapid implementation of existing law, which already requires that the API include graduation rates. Their inclusion is critical to underscoring the importance of student engagement and support in both middle and high school;
  • Greater emphasis on student achievement in science and history, to temper the overemphasis on English language arts and math;
  • A shift away from the existing API decile system (ranking schools relative to one another from 1 to 10) in favor of a scoring system pegged to an absolute standard, which creates a more accurate representation of performance.

I have worked on few issues in my legislative career that garnered more support than this attempt to ensure the state sends more appropriate signals about what it wants schools to accomplish. Republicans and Democrats, business and labor, educators and parents, law enforcement and civil rights organizations have coalesced around the need for change. We need the Governor to work with us to connect our schools to the needs of the economy we hope to rebuild in California.

Darrell Steinberg has been President pro Tempore of the California State Senate since 2008, chosen by his colleagues to that leadership post two years after he was first elected as Senator for the Sixth District representing the Sacramento area. He earlier served three terms in the State Assembly. He’s a strong advocate for education reform, children and mental health issues, and received the “John F. Kennedy Profile in Courage” national award in 2010 for his leadership in resolving the state’s 2009 budget crisis.

David B. Cohen: Why rank schools?

David Cohen
David Cohen

Imagine for a moment that California used letter grades rather than the Academic Performance Index to rate schools. If I were a parent whose child attended a high school with a “D” on its state report card, I would be gravely concerned that this school would fail to provide my student with the skills to succeed in college, and a college education is vital to my child’s future. If I had a choice, I would certainly want to move my child to an “A” school. I know these report cards aren’t perfect, but there must be a world of difference between the “D” and the “A” rankings, right? And if the “A” school was also listed among Newsweek’s Best High Schools, so much the better, I’m sure.

Wrong. The “D” school is better.

Or to be more precise, the “D” school is better if the measure of quality is college preparation. Don’t believe me? Take a look at this study – “College- and Career-Ready: Using Outcomes Data to Hold High Schools Accountable for Student Success” – from Florida. Writer Chad Aldeman sums it up this way: “While [the “D” school] got dismal marks from state and federal accountability schemes, it was actually quite successful in a number of important ways. It graduated a higher percentage of its students than [the “A” school] and sent almost the same percentage of its graduates off to college. Once they arrived on college campuses, [the “D” school] graduates earned higher grades and fewer of them failed remedial, not-for-credit math and English courses than their [“A” school] peers.

In other words, D-rated [High School] was arguably doing a better job at achieving the ultimate goal of high school: preparing students to succeed in college and careers. But because Florida’s accountability systems didn’t measure college and career success in 2006, nobody knew.

The study concludes, as you might anticipate, with a call for more data going into accountability systems, and it’s hard to argue with that. But the catch is that any rating or ranking is going to miss something, and is going to create simplistic lists of winners and losers out of what should be a more complex view of school quality.

It is time to distinguish between having data and claiming to know what it means. If we were conducting chemical experiments, it might be different. With schools, we are “measuring” extended periods of highly complex interactions among hundreds or thousands of people (different combinations of people every year), operating under different combinations of influences, and we have yet to agree as a state or society about the outcomes that matter most in that complex setting.

Ultimately, I would argue that the state should be in the business of providing resources and guidelines, and leaving the final assessments of quality and success to professional and local agencies. These agencies must ensure transparency and protect the interests of all stakeholders. They should be comfortable examining widely varying types of data and appreciating the value of each. Their judgments and conclusions would be informed by data and observations, but expressed in words – reports that don’t hide behind the false certainty or pseudo-objectivity of final scores, points, grades, or gold stars.

California high schools already engage in an accreditation process similar to that description, carried out by the Western Association of Schools and Colleges. Why not make it more meaningful, but less intensive, and expand the approach to other levels?

If our citizenry can’t handle that shift, then we have a goal for our educational system, not to produce citizens, media, and political leaders who would prefer to have a meaningless “A” or “D” slapped on a school, rather than understand and express the complex realities of school quality.

David B. Cohen is a National Board-certified teacher in Palo Alto, where he teaches high school English. He helps to direct Accomplished California Teachers and writes for the group’s blog, InterACT.

Fred Jones: Hold schools accountable for reality

Fred Jones
Fred Jones

For good or ill, California’s K-12 public education system is driven by what Sacramento – and to a growing extent D.C. – requires, funds, and measures. The “measure” driver has led to the axiom: If it isn’t tested, it isn’t taught.

But the current fixation on a narrow bandwidth of ELA and Math via fill-in-the-bubble standardized tests has not proven to be a meaningful gauge of a school’s overall performance. Moreover, it has led to the narrowing of curriculum that so many have railed against.

We should be expecting much more from schools as they strive to prepare their students for successful lives.

Many have chosen to jump on the “college for all” bandwagon, feeling this is a higher means of holding schools accountable. We have seen many districts require the UC ‘s A-G coursework of all of their secondary students.

But college should not be considered an end unto itself. In this era of dwindling public resources and exploding student debt, college should more appropriately be considered merely a means to an end: one that provides students – and the taxpayers who subsidies them – the disposition, skills, and knowledge to provide a return on the private and public investment.

There is a growing chorus of intellectuals, industry leaders, and loan-conscious parents who have begun to question the financial returns of college. Regardless of the merits of those arguments, the economy clearly does not demand that all workers have 4-year degrees.

So what shall we hold schools accountable for delivering to every K-12 student? And how do we measure that?

In his veto of SB 547, Gov. Brown acknowledged the difference between quantitative data streams and qualitative considerations, and the difficulties in measuring the latter, often more meaningful outcomes.  Paradoxically, his veto actually undermined the effort to get a more relevant accountability system.

SB 547 was a good-faith effort to broaden the accountability matrix. It sought to include more than just standardized test scores, while attempting to keep the additional criteria objectively quantifiable.

Such additional criteria would have included a school’s performance in adequately preparing students for postsecondary education opportunities, access to career planning and training coursework, dropout rates, and other substantive and serious considerations. Einstein’s quip that “not everything that counts can be counted, and not everything that can be counted counts” certainly applies. Schools must begin to report what needs to be counted to adequately measure true success.

Fred Jones has nearly 20 years of policy experience in the State Capitol as both a legislative staffer and, since 2000, as a registered lobbyist and legal counsel to several education-related clients. His primary CTE-related client is the California Business Education Association, which is also a founding member of Get REAL California, a coalition of employers, labor groups, educators, and others concerned about CTE in California schools.

Bill Honig: Provide information for school improvement

Bill Honig
Bill Honig

The first crucial question to be answered is what is the purpose and context of the measurement. Is the emphasis primarily on using test-based results for state accountability and intervention for low performance? Or is measurement primarily used as part of a broader strategy to provide useful information to schools and districts to help continuously improve teaching and learning while still supplying information to the public about school success? This second strategy requires a shift in emphasis from penalties and interventions to building a sophisticated local and state infrastructure to support school-site team building, coaching, and professional development.

The former “test with consequences” strategy rests on the assumption that setting standards, testing results, and penalizing low-performing schools, by itself, will cause major improvements. This approach does produce some beneficial results, but by neglecting the investment in building the capacity for growth, the overall effect has been found to be limited.[1] This strategy also engenders significant negative side effects such as narrowing the curriculum, lowering morale, and encouraging staffs to game the system. All the international world-class performers, as well as U.S. states such as Massachusetts and highly successful California districts such as Long Beach, Sanger, and the charter school network Aspire, have pursued the latter, more powerful, capacity building strategy.

Gov. Brown has warned of the danger of over-relying on narrow high-stakes testing in his quest to broaden measurement and the way it is used. We should explore his suggestion that the state develop local peer review as one method of feeding back useful information to guide continuous improvement.[2]

The second key question is what kind of measurements help drive the system in the right direction? Relying too heavily on reading and math or low-level multiple-choice tests has been problematical. It has motivated legislative leaders such as Sen. Steinberg to pursue legislation to broaden California’s Academic Performance Index both for accountability and instructional feedback. The API is a useful measure, but I agree that it should be broadened and deepened:

  • API does test a broad array of subjects at the high school level and some at middle grades, but needs to cover history, science, civics, and art in a more profound way, especially at the elementary and middle-grade level. This can be done in several ways. The weighting given to these subjects should be examined. Currently at the elementary level reading and math are weighted at 94%, science at 6%, and history at 0%. At middle grades it’s not much better – 85% reading and math, 7% history, and 7% science. These weights directly contribute to a narrow curriculum.
  • The state needs to add history, art, and more science to the elementary level tests, or at least embed those subjects in the language arts and math sections of existing tests, and add civic understanding assessments to the high school level.
  • While the new tests for California being developed by the SMARTER Balanced group will move away from over-relying on multiple choice for reading and math, I would also add matrix sampling of those and other subjects to the individual tests so that a broader curriculum and deeper learning, such as the ability to write essays or develop a science project, can be assessed more efficiently.
  • At the high school level, one major change would be to explore how to hold schools accountable not only for the number of students meeting A-G requirements but also for how many students at least qualify for entering a tech-prep program at community colleges. The  API would apply to a broad range of students: dropout rates, 4-year college prep rates, tech-prep rates, and course performance. I would also add some measure for the advanced students such as the number of AP courses passed.

Bill Honig began his career in education as an elementary school teacher before becoming a California State Board of Education member and district superintendent. He was elected in 1982 to serve the first of three terms as California Superintendent of Public Instruction. He subsequently published “Teaching Our Children to Read” (Corwin Press) and founded the Consortium on Reading Excellence (www.corelearn.com), which helps  schools, districts, and states implement best practices in reading and math. He is a Bay Area native, father of four, and grandfather of five.

Jeff Camp: Schools must produce an economic return (broadly)

Jeff Camp
Jeff Camp

[NOTE: An article posted here earlier today was a draft and not intended for publication.  This is the correct article.  We apologize for error]. The success of schools must not be our primary concern. Schools, after all, are only a means to an end. The center of the proverbial target is simpler, but even more difficult: prepare EACH child for adulthood.

The effort to provide opportunity for each student is a costly undertaking, and public education is its biggest component. Spending on universal K-12 education in California adds up to about $65 billion annually when all the sources (state, local and federal) are counted. To put this number in human context, taxpayers in California invest on the order of $140,000 in each student’s thirteen years of K-12 education – roughly equivalent to paying about two dollars above minimum wage for every hour a student spends in class.

As with any big investment, success must be measured in terms of Return on Investment (ROI). Measuring success on these terms requires long-term data about each student’s long-term success, viewed broadly and over a time frame spanning decades, not just school years.

What is the long-term economic payback on that $140,000 investment for each student? Today, we don’t really know. Evaluating the return requires estimating both value produced and costs avoided. Education produces value by helping each student find his or her place in the world, including work that earns enough to pay taxes. Education avoids costs by helping students grow into self-supporting, resilient and law-abiding adults. Our system is set up to track neither.

For the last decade, the Academic Performance Index (API) has been the dominant tool for summarizing a school’s performance in California. This score, distilled annually from a changing assortment of annual tests, serves as a shorthand metric of academic achievement at the school and district levels by grade level. Unfortunately, the API only measures the academic success of those who show up. If every struggling student in a school were to drop out, the API score for that school would, perversely, rise.

The state’s system of measurement for education should be built online, in a manner that allows students to show what they know regardless of their nominal grade level. If this seems like whimsy, take a look at the coaching module of Khan Academy for an early example of what the future of measurement may look like, at least in math.

The public has grown accustomed to the idea that products and services should be evaluated, rather frequently, and that evaluation should lead to action. In order to sustain public support for investing in education, California needs to make a set of serious investments to systematically provide everyone involved with better, more personally useful information over a more meaningful arc of time. We rely too much on summary numbers partly because that is all we have at present. California should do better.

For starters, California should invest in modern data systems to track and support investments in human development including education. In the age of Facebook, it is no longer OK for California’s education system to operate with outmoded data systems.

California needs a platform that usefully connects parents, students and teachers, including accurate data to inform the work they do together. This is not an investment that each district can or should pursue on its own; it is far too difficult, much too important, and frankly its implications extend beyond education.

Jeff Camp chairs the Education Circle of Full Circle Fund, an engaged philanthropy organization cultivating the next generation of community leaders and driving lasting social change in the Bay Area and beyond. He is the primary author of Ed100.org, a primer on education reform options in California. Since leaving a career at Microsoft to work for education change, Jeff has served on multiple education reform committees including the Governor’s Committee on Education Excellence.

Leave a Reply

Your email address will not be published. Required fields are marked *