Steinberg’s API alternative

Rebuffed by Gov. Jerry Brown last year, Senate President pro Tem Darrell Steinberg is back with another academic accountability bill, this time giving the governor lots of latitude to help redefine how to measure schools’ performance. SB 1458 needs to be vague, because, at this point, no one but Brown professes to know what he has in mind.

Recognizing that the Academic Performance Index, based predominantly on English and math standardized test results, was too narrow a gauge, Steinberg last year proposed replacing the API with an Education Quality Index that would have included other indexes, such as dropout rates, the need for remediation in college, success with career technical education programs, and graduation rates. Standardized tests would have counted no more than 40 percent of the EQI in high school. Steinberg and key supporter Superintendent of Public Instruction Tom Torlakson built an impressive coalition of supporters – business and civil rights groups, career and technical education groups, charter schools, the state PTA, and early childhood education advocates.

But in his veto message of SB547, Brown criticized the continued reliance on quantitative measures. “SB 547 would add more things to measure, but it is doubtful that it would actually improve our schools. Adding more speedometers to a broken car won’t turn it into a high-performance machine.”

In the new bill, Steinberg would retain the 40 percent maximum use of the API, and would instruct Torlakson to expand the use of science and history tests within it. As for the remaining 60 percent, SB 1458 would allow Torlakson and the State Board of Education to incorporate another idea that Brown mentioned in his veto message and State of the State message: school inspections or visitations to measure the quality of learning and instruction not measured by standardized tests. Brown has not clarified if he is talking about a corps of outside inspectors, as is used in England by the Office for Standards in Education, Children’s Services and Skills (Ofsted) or a less formal system of intradistrict inspections.

The State Board, led by Brown advisor Michael Kirst, plans to make the adoption of new accountability measures a priority this year. At his instigation, the San Francisco-based nonprofit, nonpartisan research organization WestEd has surveyed other states’ accountability systems and analyzed all of the data that the state collects. Its report has not yet been released.

The timing is right for a new accountability system. Brown proposes to give near-total control over spending and budget decisions to local districts this year, as part of his school finance reforms. It will become imperative to  create better ways to measure whether schools are providing a rich environment for learning, spending dollars effectively on students who have been targeted for extra money, and preparing students well for post high school jobs and colleges.

Common Core, ‘dubious causality’

The horse race of international rankings in education is based on misconceptions that can lead countries such as the United States to consider sweeping reforms that probably won’t improve academic achievement, according to a new report. The 2012 Brown Center Report on American Education released yesterday by the Brookings Institution makes a case against Common Core standards – arguing that California’s current standards are superior – and cautions against placing too much weight on the National Assessment of Educational Progress (NAEP) and international comparisons.

“We have to be careful when looking at test score data; it’s not the same thing as how many points did the New York Giants score versus the New England Patriots in the Super Bowl. These tests have to be interpreted very carefully,” said author Tom Loveless in a video accompanying the study.

Loveless is especially critical of using international exams, such as PISA, to rank countries’ educational systems. The United States tends to score in the average range on the test, behind top performers including Shanghai, Finland, Singapore, and Canada.

In what Loveless calls a flaw of “dubious causality,” lower performing countries mistakenly look for a single policy to explain the success of top performers. One of those dubious connections he’s referring to is Common Core standards. He said advocates of Common Core often point out that the top ten countries all have national standards. But, said Loveless, “if you look at the bottom ten nations in the world, they all have national standards too.”

The report cites arguments by two outspoken critics of Common Core in California, Ze’ev Wurman and Bill Evers, who “conclude that the math standards, in particular, are inferior to existing standards in Massachusetts and California.”

Wurman was a member of the Mathematics Curriculum Framework and Criteria Committee that developed California’s 1997 mathematics framework, and Evers served on the 1996 California State Commission for the Establishment of Academic Content and Performance. Both were members of the California State Academic Content Standards Commission and, as TOP-Ed reported here, their fellow commission members overwhelmingly rejected their efforts to rewrite the Common Core standards to look like California’s earlier math standards.

One of those other commissioners is Scott Farrand, a math professor at Sacramento State University.  He questioned how California’s standards can be considered the highest in the country when two-thirds of elementary students score advanced or proficient on the California Standards Test, but that falls to less than a quarter by eleventh grade. What that says to Farrand is that merely setting a high bar doesn’t improve achievement.

“What sets the Common Core State Standards apart is not the level of the standards, however one might measure that. It is their focus and coherence, and their insistence on student understanding,” said Farrand.  He’d like to see the “my standards are higher than yours” posturing end so the people responsible for implementing Common Core standards in California can spend their time understanding “what standards can and should do,” rather than engaging in “silly bickering” that detracts from that progress.

Brown wants fast results, fewer tests

Gov. Jerry Brown picked up on a common complaint of teachers, superintendents, and parents in his State of the State address last week: Schools have closed for the summer by the time they get the scores of standardized tests that students took the previous spring. At that point, the results are a lot less useful.

Brown’s call for a quicker turnaround – “I believe it is time to reduce the number of tests and get the results to teachers, principals, and superintendents in weeks, not months” – is doable, according to experts I spoke with. Most states already get their tests back quicker than California, and California can do so, too – if the state is willing to change how it administers and validates its standardized tests. The state could have – and probably should have – taken this step years ago, said Doug McRae, a retired standardized test publisher and occasional TOP-Ed contributor.

In his speech, Brown pointed to one benefit of earlier results: “With timely data, principals and superintendents can better mentor and guide teachers as well as make sound evaluations of their performance.” Given Brown’s skepticism of standardized tests, I was surprised he raised linking them to teacher evaluations. His friends at the California Teachers Assn. must have cringed. But as an analytic tool, teachers do need the CST results before they depart for the year and can think over the summer of the changes they’ll make. Once they return in the fall,  they have next year’s students on their minds.

A quicker turnaround of CST results can also help teachers and schools make wiser placement decisions for students in courses like Algebra I. And it can be helpful to students, too, especially high school juniors taking the Early Assessment Program, the college readiness exam that’s part of the 11th grade English and math CST. Early results can guide them on what to do their senior year. And if, as some suggest, the state starts counting end-of-the-year subject exams toward a student’s grade (either that or stop giving some altogether), then they’ll have to be graded and returned sooner.

Change in testing method

CSTs are administered after 85 percent of the instructional year is over. Since each district sets its own calendar, and some districts on year-round schedules start as early as July, CSTs are taken from February through May in parts of California. The state could administer the tests sooner, after, say, 75 percent of instructional days, says Rachel Perry, director of the Analysis, Measures and Accountability Reporting Division for the state Department of Education. But that would create comparability problems with past years’ results.

The more practical option would be to change the way test results are compared yearly, from a “post-equating” to a “pre-equating” method. Under the current system, the test publisher, Educational Testing Service, waits until after the tests are administered to do a scoring analysis of the new questions that were introduced. The alternative is “pre-equating,” in which new items would be introduced and analyzed as sample items in earlier years. Then the turnaround would take weeks, not months. The public release of statewide CST and individual school results would remain Aug. 15.

McRae said it was probably smart to have been cautious and to have used post-equating in the early years of the CSTs. The state could have switched methods once it had experience under its belt, he said. “There’s no longer a risk.”

Perry said the state could convert to a quicker turnaround by the 2013-14 school year, but this should be done in the context of a larger plan for testing. Brown has also called for fewer standardized tests, and the State Board of Education must decide which tests may be displaced by new assessments connected to the Common Core standards in English language arts and math.

“We need a comprehensive look at the entire system,” said Perry.

Final exams

It’s not just any Friday; it’s the Friday before the biggest holiday week of the year. So if you’re reading this, you’re probably trying to escape from a) out-of-town guests, b) another tin of homemade cookies that reminds you about that resolution to join a gym, c) the annual office party that’s been downgraded by the economy from a formal soiree with a band and open bar to cheese Danish and coffee, or d) all of the above.

Hmmm, multiple choice. That gives us an idea for something that will keep TOPed in your thoughts next week while we take a break. We’ve compiled an array of questions, released by the publishers, from a variety of tests. Pour yourself a glass of eggnog – no scrimping on the nutmeg – power up your graphing calculator, and open your booklet.
[Click here for answers]

California High School Exit Exam

Mathematical Reasoning

1.    The table below shows values for x and corresponding values for y.

table

Which of the following represents the relationship between x and y?

A.        y = 1/7x

B.        y = 7x

C.        y = x-6

D.        y = x-18

Algebra I

2.    Which of the following is equivalent to 1-2x>3(x-2)?

A.        1-2x>3x-2

B.        1-2x>3x-5

C.        1-2x>3x-6

D.        1-2x>3x-7

Statistics, Data Analysis, and Probability

3.    The Smithburg town library wanted to see what types of books were borrowed most often.

­­
pie chartAccording to the circle graph shown above –

A.  More Children’s books were borrowed than Romance and Science Fiction combined.

B.  More than half of the books borrowed were Children’s, Mysteries, and Art combined.

C.  More Mysteries were borrowed than Art and Science Fiction combined.

D.  More than half of the books borrowed were Romance, Mysteries, and Science Fiction combined.

English-Language Arts

maya start Maya 1

maya
maya 8

California Standards Test

Grade 8/History-Social Science
CST grade 8 History-social science 1

California Standards Test

Grade 5/Science

CST science grade 5
National Assessment of Educational Progress (a.k.a., the Nation’s Report Card)

Grade 12/Economics

NAEP grade12 econ

Programme for International Student Assessment (PISA)
Administered every three years to 15 year olds in more than 70 countries, including the United States

Math

The approximate distance to stop a moving vehicle is the sum of:

– the distance covered during the time the driver takes to begin to apply the brakes
(reaction-time distance)

– the distance travelled while the brakes are applied (braking distance).

The ‘snail’ diagram below gives the theoretical stopping distance for a vehicle in good braking condition (a particularly alert driver, brakes and tyres [tires] in perfect condition, a dry road with a good surface) and how much the stopping distance depends on speed.

snail

QUESTION 1: BRAKING

If a vehicle is travelling at 110 kph, what distance does the vehicle travel during the driver’s reaction time?

QUESTION 2: BRAKING

If a vehicle is travelling at 110 kph, what is the total distance travelled before the vehicle stops?

QUESTION 3: BRAKING

If a vehicle is travelling at 110 kph, how long does it take to stop the vehicle completely?

California Basic Educational Skills Test (CBEST)
Must be passed by all California teachers before starting their certification program.

Reading

cbest

Math

cbest math

Writing

cbest writing

Brown skeptical of key ed bill

Gov. Jerry Brown has warned lawmakers that his veto pen will flow freely over the next three weeks. Among bills on the threatened list is potentially the most far-reaching K-12 education legislation before him – a bill that would significantly shift the state’s accountability system away from its concentration on standardized tests. SB 547 is also a priority of its author, Senate President pro Tem Darrell Steinberg.

Acknowledging that the bill may be in trouble, Steinberg said Friday that Brown “has expressed some doubts” about it, while not precluding the possibility that he might sign the bill, in several discussions they have had.

Brown, too, has expressed a dislike of the reliance on standardized tests by the state and particularly the federal government with the No Child Left Behind law to measure the success of students and teachers. So one would think that he would be simpatico with Steinberg on SB 547, which would create an Education Quality Index, or EQI,  to replace the Academic Performance Index, or API, with new indices to downplay standardized tests.

Brown’s apparent objection isn’t about the bill’s cost but instead about uncertainties over a possible demand for new data. Brown has a visceral distrust of statewide data systems and use of data in general. He killed money for a statewide database on teachers – CALTIDES – and wanted to delete additional federal funding for the state’s database on students, CALPADS, though the Legislature reinserted it into the budget.

SB 547 would create new indices measuring a broader range of student achievement, such as career and college readiness, accomplishment in areas outside of core subjects, and high school graduation and dropout rates. For high schools, standardized tests would be a maximum of 40 percent of the new EQI; for middle and elementary schools, standardized tests would comprise a minimum of 40 percent of the EQI.

“This bill is consistent with his (Brown’s) philosophy of getting away from test scores,” Steinberg said.  And with Congress deadlocked over the reauthorization of the Elementary and Secondary Education Act, the formal name for No Child Left Behind, California should set its own priorities and “lead by example,” Steinberg said.

Superintendent of Public Instruction Tom Torlakson and an advisory committee would develop the EQI, but the State Board of Education, whose members are Brown appointees, would have to approve it. That would give Brown control over the indices, Steinberg said. Rather than veto the bill, Brown could sign it with a message signaling the changes he would want to the bill next year, he said. (He also credited Brown for spending a lot of time and thought on the bill.)

Meanwhile, Steinberg has been campaigning to raise visibility for the bill; last week, he and Torlakson held a press conference in Los Angeles with Torlakson, U.C. Regent George Kieffer, and representatives from business and civil rights groups, the state PTA, and early childhood education advocates to call on Brown to sign it.

The bill has substantial support from diverse groups (see list at the end of Steinberg’s fact sheet), representing business and manufacturing, career technical education, gifted students, charter schools, school administrators, and school boards. The California Teachers Association hasn’t taken a position on SB 547; the California Association of School Counselors and the California Business Educators Association are backing it. Steinberg is hoping other groups will speak up between now and Oct. 9, the last day for Brown to sign or veto bills.

John can be reached at john@svefoundation.org.

Lawmakers advance standards

(Kathy and John combined efforts on this post.)

Heading into the final week of the session, the Legislature has sent bills to Gov. Brown that would revise state science standards and build a bridge to the approaching Common Core with instructional materials, curricula and professional development.

California will revise K-12 science standards for the first time since they were adopted 13 years ago, a light year in a fast-changing world.

The Assembly passed yesterday and forwarded to Gov. Brown SB 300, which will authorize Superintendent of Public Instruction Tom Torlakson to appoint a committee of experts, including elementary and secondary science teachers, school administrators, and university professors. Under a tight timeline, they will present the revised standards to the State Board of Education by March 2013, in order for for the Board to modify and adopt them by July 30, 2013, four months later. There will be at least two public meetings before then at which the public can comment on the standards.

SB 300, sponsored by Democratic Sen. Loni Hancock of Berkeley, was written by the California Science Teachers Association, which has been calling for standards revisions for years – and not only because they were outdated, without mention of stem cells and biotech. Many teachers have argued there are too many science standards, leading to too little opportunity for in-depth science exploration and little understanding of scientific concepts.

The division of opinion dates back to 1998 and a bitter split between scientists who favored an inquiry-based or hands-on approach to science education and those focused on a content-based curriculum. The latter, led by then 86-year old Nobel Prize winning physicist Glenn Seaborg, won out, and California’s science standards reflect that philosophy.

The pendulum is already swinging. SB 300 directs the new standards to be based on the Next Generation Science Standards, which will be the science version of the Common Core standards, a multistate effort, led by Achieve Inc. The standards will be an elaboration of the Framework for K-12 Education, written by the Board of Science Education of the National Research Council. In an interview in TOPed last month, the chairwoman of the board, Stanford physicist Helen Quinn, said that the new standards will provide a “coherence” and integration of core scientific ideas over multiple years that have been missing in the current state standards. The standards, she said, will focus on “crosscutting concepts” that stress similarities in the scientific method and approaches – analyzing data, developing models, defining problems, carrying out investigations – common to physical science, biology, and engineering.

California is competing to be named among a handful of states that will work with Achieve to create the standards. The winners were to be announced in August.

Among the critics of the new Frameworks is Ze’ev Wurman, a software engineer from Palo Alto and former adviser to the U.S. Department of Education who helped write the state’s math curriculum frameworks. Wurman fears that the new standards will be light on actual science and heavy on science appreciation. The frameworks did not call for the application of mathematical equations and techniques; the lack of integrating algebra and trigonometry would appear to be a fundamental flaw that will produce “good consumers of science and technology,” rather than prepare them for training in actual science, Wurman wrote in a blog entry.

A head start on SMARTER Balanced

Lawmakers also sent Gov. Brown the first of three bills aimed at keeping the state one step ahead of planning for the coming of Common Core.

SB 140 by State Senator Alan Lowenthal (D-Long Beach) requires the State Department of Education and State Board of Education to compile a list of supplemental instructional materials for math and English language arts in elementary and middle school.

It’s a stop-gap measure to make sure students are prepared for the new tests, which will cover Common Core standards. Those exams could start in 2014, but the state isn’t scheduled to take up a full-blown textbook adoption until sometime after 2015.

An early iteration of the bill nearly died in a disagreement over eighth grade math. The State Board of Education last year approved two sets of math standards for grade eight: Common Core and Algebra I. SB 140 tried to include materials for both, but ran into several roadblocks. Critics said the dual standards could lead to tracking and revert to a time when schools had two sets of expectations, often based on race, ethnicity, or income.

The dispute was settled by removing eighth grade math from the bill. The final version covers English language arts for kindergarten through eighth grade and math for kindergarten through seventh grade.  State education officials want the State Board of Education to take a redo and adopt a single eighth grade math standard.

The California Department of Education has already started soliciting materials from publishers. Once that’s finished, the final list will go to the State Board for approval. But because these are supplemental materials, local districts aren’t restricted to the State Board’s list the way they are with textbook adoptions. Districts can choose their own materials as long as they cover the standards, or they can choose to do nothing.

Companion bills moving along

Two sister bills still in the legislative process would round out early preparation for the Common Core standards.

AB 250, by Assemblywoman Julia Brownley (D-Santa Monica), adds professional development and requires the State Board to adopt new curriculum frameworks and evaluation criteria that are aligned to the Common Core academic content standards. It also keeps the state’s STAR testing system in place for an extra year, when it will be replaced by the new assessments developed for Common Core.

Assemblyman Felipe Fuentes (D-Sylmar) has the third leg of the stool. His bill, AB 124 , requires the state to convene a group of experts in English language instruction to ensure that the curriculum, materials, and assessments for English learners are aligned with the Common Core standards.

The two bills won’t come up for a floor vote until Monday at the earliest.

Test scores up – not enough for feds

California schools had their best year yet toward meeting their targets on the Academic Performance Index (API), the state’s ranking system. So it’s puzzling why thousands of schools could face sanctions for not meeting federal proficiency levels. Trying to make sense of the complicated formulas that cause this discrepancy is like being a kid in a Peanuts comic strip listening to the teacher say “Wah wah wah.”

In one corner, we have nine years of continued gains on the California Standards Tests, with a record 49 percent of the state’s schools meeting or exceeding the API target of 800, according to scores released Wednesday by the state Department of Education.

“At school after school, and among every significant ethnic group, California’s students are performing better than ever,” said State Superintendent of Public Instruction Tom Torlakson, “even in the face of severe cuts to school funding.”

In the other corner are 913 schools about to join nearly 4,000 others already in Program Improvement status for failing to make large enough gains under the federal accountability system known as Adequate Yearly Progress. But here’s the rub: Many of the schools in Program Improvement and many of those succeeding in the state system are one and the same.

“We believe the No Child Left Behind policies are flawed,” said Torlakson during a call with journalists. “Despite 20- to 30-point gains, they [the schools] will be dubbed a failure; it doesn’t make sense.”

Torlakson sent a letter to U.S. Education Secretary Arne Duncan last week seeking a waiver from designating anymore schools as failing to meet AYP. [Read our article here.]

William Habermehl, Superintendent of the Orange County Office of Education gave his full support to that request.  On the same call with Torlakson, Habermehl said, “We need to let Washington know that they need to return the control of public education back to the states and the local levels and stop this nonsense.”

Different rulers and different measures

California’s Academic Performance Index is based on a scale of 200 to 1,000. The state has set 800 as the target for all schools to meet. According to the API results, 55

Source: California Department of Education (Click to enlarge)
Source: California Department of Education (Click to enlarge)

percent of California’s elementary schools hit 800, while 43 percent of middle schools did so. At the same time, just 35 percent of elementary schools and 18 percent of middle schools met the federal targets.

The reason the state and federal results are so far apart is that API rankings are a composite of all student scores in each school. Although the State Department of Education does disaggregate test scores by race, ethnicity, language, and other traits, those aren’t factored into the API score.

The federal government has a higher bar. Under No Child Left Behind, every one of those subgroups must reach the proficiency level on the standardized exams in order for the school to pass.

But high schools are a slightly different beast.

They reversed the trend with just 28 percent reaching 800 on the API and 41 percent achieving Adequate Yearly Progress.

“Not only is the measuring stick different, it’s measuring something different,” explained Rachel Perry, director of assessment and accountability for the California Department of Education.

The API for high schools is a combination of California’s standards-based test scores and results of the California High School Exit Exam.  But Adequate Yearly Progress is based just on exit exam scores, and, as we reported here last week, passing rates are increasing.

Here come the caveats

The state’s information guide on Adequate Yearly Progress runs 81 pages.  Half-way through, on page 41, is the section on Safe Harbor.  It would take an entire article to describe all the conditions of Safe Harbor – and we plan to do that very soon – but the gist of it is that it gives schools a second chance to reach the NCLB pass level if subgroups have shown improvement on test scores.

Another point of possible distortion is the increasing number of special needs students taking California’s alternative test, the California Modified Assessments.  One of our regular Op-Ed contributors has a column on that today.

Then there’s the issue of what it means when a school reaches 800 on the API scale.  The state Department of Education puts so much stock on the number that it’s taken

Source:  California Department of Education. (Click to enlarge)
Source: California Department of Education. (Click to enlarge)

on a magical aura of success.  But that’s not the case.  As the chart on the right shows, each level, from far below basic to advanced, is assigned a numeric value.   So a score of 800 doesn’t indicate that all the students in a school are proficient or better, just a little over half of them.

“Your API score doesn’t necessarily give you information about a subject area or grade level,” said Perry.  “That’s part of the difficulty of the index. But it’s part of the beauty, too, because you know students are doing better.”

Expansion of test for special ed students distorts latest API results

The 2011 California Academic Performance Index (API) results were released yesterday, and revealed slow but steady gains compared to previous years’ results. However, changes in California’s Standardized Testing and Reporting (STAR) program have led to artificial increases in API gains over the past four years. Sorting out how much of the “good news” announced yesterday is based on true academic achievement gain made by California kids and how much may be due to changes in the STAR program is not a trivial issue.

The numbers. If we record statewide API results for the last four years by grade span, including the results announced today, we find the following (see API chart):

API results (click to enlarge)
API results (click to enlarge)

Overall, the gains for 2011 appear to be marginally less than the reasonably steady gains recorded in 2008, 2009, and 2010. The multiyear gain scores from 2007 to 2011 are more noteworthy: 47 points for grades 2-6 and 58 points for grades 7-8. The question is: Are these multiyear gains true achievement gains?

The program changes. In 2008, California began introducing the California Modified Assessments (CMAs) to the STAR program for selected Special Education students. The CMAs have been phased in over four years – for grades 3-5 in 2008, grades 6-8 in 2009, and grades 9-11 over two years in 2010 and 2011. The CMAs are easier tests than the mainstream STAR California Standards Tests (CSTs), designed for Special Education students who score very low on the mainstream CSTs. Special Education students who scored Far Below Basic or Below Basic on a STAR CST in the previous year are eligible to take a CMA, with final determination for which test to take left to the student’s Individualized Education Program team. The CMAs were initiated in response to a federal flexibility program, and were to be targeted for no more than 2 percent of overall enrollment, or roughly 20 percent of Special Education enrollment.

As documented in a TOPed post, initiation of the CMAs over the past four years has affected STAR statewide summary data reported by the State Superintendent. By systematically eliminating students who score Below Basic or Far Below Basic from the STAR CST program, the percentages of students who scored Proficient or Advanced on CSTs have been artificially increased by an estimated 26 percent over four years. In addition to the problem of accuracy for summative data for STAR, the TOPed post also documented the fact that CMA usage has increased far more rapidly than anticipated and has far outstripped the anticipated 2 percent of overall enrollment or 20 percent of Special Education enrollment.

For grades 3-8, where CMAs have now been used for several years, CMAs were administered to 5.2 percent  of the total number of students tested in grades 3-8 in 2011, and more CMAs than CSTs were administered to Special Education students. Finally, when one looks at CMA usage by district, one finds a wide discrepancy among districts in the percentage of Special Education students administered the CMAs – from less than 20 percent for some districts to almost 70 percent for other good-sized districts. For a sampling of wide differences, see the chart for Santa Clara County.

in Santa Clara County, grades 3-8. The CAPA entries reflect estimates based on Levels II-IV only. The Santa Clara Co numbers are taken from STAR database and do not reflect column sums. The numbers are taken from the STAR database released 8/15/11. Chart by Doug McRae (click to enlarge)
The numbers in the chart are for districts in Santa Clara County, grades 3-8. The CAPA entries reflect estimates based on Levels II-IV only. The Santa Clara Co numbers are taken from STAR database and do not reflect column sums. The numbers are taken from the STAR database released 8/15/11. Chart by Doug McRae (click to enlarge)

The crux of the issue for the 2011 API results is that the introduction of easier tests for a significant portion of Special Education students generates inflated API multiyear scores. If the 2 percent limit originally envisioned had been strictly applied to API calculations, the artificial increases in API scores would likely not be a major factor.  However, with the CMA usage running 2 to 3 times higher than anticipated, the artificial increases in API scores are quite notable. Also, CMA now becomes a potential way for local districts to game the API system, testing more Special Education students with CMA to artificially increase their API scores.

The Big Picture. I discussed the larger policy implications for the introduction of CMAs to California’s STAR statewide assessment system in a TopED commentary posted earlier this year. In that commentary, I note the issue is not the idea of a modified assessment – I agree that CMAs yield better data for selected Special Education students than counterpart CSTs.  The issues raised here are (1) How should CMA scores be treated for API calculations? and (2) Are CMAs being overused and/or abused by local districts to artificially raise API scores?

The API Inflation Effect. How much are the API scores inflated?  It is relatively easy to estimate this; a year ago I computed the API inflation effect for the 2010 API scores; it was in the 35 to 40 percent range for grades 3-8. With the same methodology used a year ago, I’ve computed the inflation effect for the 2011 API scores due to the introduction of the CMAs over the past four years. Since the CMA performance levels for all tests grades 9-11 have not been set as yet, it is possible to estimate the inflation effect only for grades 3-8 at this time. The results of the 2011 API inflation effect  are (see CMA inflation chart):

CMA inflation rate (click to enlarge).
CMA inflation rate (click to enlarge).

The bottom line here is that the statewide API increases reported for the past four years for grades 2-6 have been inflated by 42.4 percent, and the statewide API increases reported for the past three years for grades 7-8 have been inflated by 34.4 percent. Stated another way, more than a quarter of the statewide gains claimed by the SPI/CDE over the past 3-4 years have been artificial gains due to changes in the STAR program (i.e., introduction of CMAs) rather than true achievement gains.

Counterarguments/Remedies. When I raised the issue of inflated API statewide gain results last year, CDE staff presented two counterarguments:

  • Initially, CDE staff claimed 2010 API results were not inflated; instead, API scores in previous years were deflated due to the administration of inappropriate tests to selected Special Education students. In other words, the previous years’ CST scores were not valid while the current year’s CMA scores are valid. However, even if this argument is true (and I do not challenge it from an individual student point of view), the gains in API scores over multiple years will be inflated unless either the “deflated” API scores from previous years or the “inflated” API scores from this year are adjusted so that there are apples-to-apples comparisons to calculate gains.
  • CDE staff claimed that appropriate adjustments are made for each year’s Base API to Growth API calculations, to take into account the replacement of scores from the more rigorous CSTs by scores from the less rigorous CMAs. However, this analysis does not address API 1-yr Base-to-Growth results; rather it addresses multiyear API trend data. Other technical adjustments made during the API calculation process also do not address the distortion that CMAs have introduced to multiyear API gain data.

There are relatively simple remedies for the problem that California is reporting inflated API multiyear gain data. Adjusting CMA performance level scores downward for API calculations is one solution that has precedent in the API system. However, remedies need to be vetted by California’s API advisory committee and then presented to the State Board of Education for approval before they can be executed. So far, neither CDE staff nor the SPI nor the SBE has shown the leadership needed to fully vet the issues involved with the introduction of CMAs to California’s statewide STAR assessment system and to consider appropriate remedies.

Doug McRae is a retired educational measurement specialist living in Monterey. In his 40 years in the K-12 testing business, he has served as an educational testing company executive in charge of design and development of K-12 tests widely used across the US, as well as an adviser on the initial design and development of California’s STAR assessment system. He has a Ph.D. in Quantitative Psychology from the University of North Carolina, Chapel Hill.

CST results need an *

Since 2008, increasing numbers of low-performing special education students have been taking a new and easier standardized test rather than the California Standardized Tests. While it’s appropriate for many of them to take the California Modified Assessment, the exclusion of 185,000 special education students this year had the effect of appreciably boosting the statewide CST scores.

That’s been the contention of Doug McRae, a retired executive who oversaw the design of standardized tests nationwide and has raised the issue with the State Board of Education. On Monday, he once again did the calculations to show the impact.

Factoring in the CMA effect, the statewide increase in proficiency on the CSTs in 2011 would have been 1.6 percent, not 2.05 percent, as the state Department of Education has stated. And the four-year effect of diverting special education students to CMA had an inflationary effect of 26 percent, according to McRae. The cumulative increase in proficiency would have been 8.95 percent instead of 11.25 percent, had the students taken the CST instead of the CMA.

California is one of two dozen states to create a modified assessment for special education students, as permitted by the federal Department of Education. Only those students who do poorly on the CST the previous year, with a below basic or far below basic score in math or English Language Arts, are eligible to take the CMA in that subject area. The state introduced the CMA for grades 3-5 in 2008, grades 6-8 in 2009, and for grades 9-11 over the past two years. The number of students taking the CMA increased from 39,000 to 184,000 in four years (see chart).

(source: Doug McRae)
(source: Doug McRae)

The federal Dept. of Education expected that no more than 2 percent of a state’s student body would take the modified exam. This year, 4.4 percent of students took the CMA, more  than 40 percent of special education students. According to McRae, “the number of students taking the CMA far exceeds initial plans for this test, and are likely to increase to an alarming percentage of the entire group of Special Education students in California.”

(The CMA data are part of McRae’s analysis of the latest CST scores. Readers interested in reading more can find the full text here.)

Dream Act sent to governor

And they’re off!  Bills flew through the senate and assembly chambers as lawmakers wrapped up as much business as possible before leaving for summer recess on Thursday afternoon.   When they return on August 15th, the docket will still be full, but the fate of some key education bills is coming into sharper focus.  Here’s where they stand.

Civil and Equal Rights

AB 130 and AB 131: California Dream Act of 2011
Assemblyman Gil Cedillo

The state senate passed and sent to Gov. Brown the first of two Dream Act bills by Assemblyman Cedillo allowing some undocumented college students to apply for private scholarships at California’s state colleges and universities.

None of this money comes from the state budget; it’s from private donors who establish scholarships administered through UC, Cal State and community colleges.  To be eligible, students will have to meet the requirements for paying in-state tuition under AB 540, a 2001 law that applies to any student, citizen or not, who attended a California high school for at least three years and graduated or earned a GED.

The bill passed by a vote of 26 to 11 along party lines, with one exception.  Republican State Senator Anthony Cannella voted with the majority.  In a prepared statement, Cannella said, “Having an educated workforce will be critical to the future strength and health of our economy, and giving eligible high-school graduates the opportunity to apply for private scholarship funds – at no cost to California taxpayers – is consistent with this goal.”

It may also help that his district, which covers Merced, Monterey and Salinas, is more than 55 percent Latino. It also has more registered Democrats than Republicans.

Cedillo’s companion bill, AB 131, faces a tougher road.  That one would let AB 540 students apply for state financial aid through the CalGrants program.  AB 131 was placed on the senate appropriations committee suspense file and won’t be considered until late August.

Status:  On the Governor’s desk.  Gov. Brown hasn’t said whether he’ll sign AB 130, however, his spokesman says the Governor “continues to support the principles behind the Dream Act and will closely consider legislation that reaches his desk.”

SB 48:  The FAIR (Fair, Accurate, Inclusive and Respectful) Education Act
Senator Mark Leno

Gov. Brown signed this landmark bill on Wednesday, July 13, making California the first state in the nation to include the accomplishments of gay, lesbian, bisexual and transgender persons in school textbooks and instructional materials.

“History should be honest,” said the Governor in a written statement.  “This bill revises existing laws that prohibit discrimination in education and ensures that the important contributions of Americans from all backgrounds and walks of life are included in our history books. It represents an important step forward for our state.”

Status: Signed into law by Gov. Brown.

Charter Schools

AB 86Charter School Authorizing Petitions
Assemblyman Tony Mendoza

Gives classified employees a voice in creating a new charter school or converting an existing public school to a charter school.

Under the current charter school law, petitions for new charter schools need enough signatures from parents or guardians to equal at least half the number of students expected to enroll in the school during its first year, or by at least half the number of teachers expected to be hired the first year.

Mendoza’s bill gives classified employees a voice in creating new charter schools by adding their signatures to those currently required from teachers and other certificated staff (excluding administrators), that equal at least one-half the number of all those employees that the charter expects to hire.

Status: Ordered to a third reading in the senate.

AB 360: Charter Schools
Assemblywoman Julia Brownley

This bill is intended to create more transparency in charter schools by requiring charter school board meetings to be open to the public under one of the state’s open-meeting laws – the Ralph M. Brown Act or the Bagley-Keene Open Meeting Act, and it would require charter school governing boards to adopt conflict of interest policies.

Status:  AB 360 passed the state senate on July 14 and is headed back to the assembly to address some amendments.

AB 440: Charter Schools Academic & Fiscal Accountability
Assemblywoman Julia Brownley

This bill and one in the state senate by Sen. Joe Simitian covered similar ground in setting rigorous academic standards that charter schools must meet as a condition for having their charters renewed.  The legislators, along with the California Charter Schools Association, reached an agreement on accountability standards for renewal and wrote them into AB 440 and Simitian’s bill, SB 645 (see below).  In addition, AB 440  would allow school boards to consider an operator’s history of managing charter schools and whether the school’s student population reflects the demographics of the local population when deciding whether to renew a charter. It also requires charter schools to hire the same high quality financial auditors as their school districts.

Status: Awaiting hearing in Senate appropriations committee.

SB 645: Charter School Renewals
Senator Joe Simitian

The agreement with Assemblywoman Brownley and the California Charter Schools Association amended SB 645.  Now, in addition to containing the same academic accountability standards as AB 440, this bill also makes changes to the Charter School Facility Grant Program to provide assistance with facility rent and lease costs for charter schools, based on the percentage of pupils who are eligible for free and reduced-price meals.

Status: Amended in the assembly and sent back to the Assembly appropriations committee.

Community College

AB 108: Community College Fee Hike
Assembly Budget Committee

Community College students could get a reprieve from new fee hikes under this legislation.  Fees are currently due to increase from $36 per unit to $46 per unit at the start of the fall 2011 term.

This bill would allow that increase only if the state’s General Fund revenue forecast for the 2011-12 fiscal year are less than $ 87,452,500,000.  If the fee hike is necessary, it would start with the winter term, rather than the fall term.

Status: AB 108 passed the state senate on July 14, and has been sent to Gov. Brown.

AB 743: Community College Common Assessments
Assemblyman Marty Block

Each of California’s 112 community colleges uses a slightly different version of the student placement tests for math and English, and each school has its own cut-off score, the grade below which students are placed in remedial courses.

Block’s bill would require the Community College Board of Governors to establish a common assessment system.

Status: AB 743 is on the Senate appropriations committee suspense file and will be considered in August after lawmakers have completed work on all other bills.

Foster Youth

SB 578: Partial Credit for Foster Youth
Senator Gloria Negrete McLeod

Education is often disrupted for foster youth because they’re frequently moved from home to home.  Sen. McLeod’s legislation helps foster youth stay on track for high school graduation by requiring schools to grant partial credit for courses a foster child was taking in one school before being moved to a different school.

Status: Scheduled for a hearing before the Assembly appropriations committee on August 17.

AB 194: Public postsecondary education: priority enrollment: foster youth
Assemblyman Jim Beall, Jr.

This bill would require the California State University and each community college district, and requests the University of California, grant priority registration for classes to foster youth and former foster youth.

Status:  Placed on the Senate appropriations committee suspense file to be considered after lawmakers have completed work on all other bills.

AB 709: Foster Children:  School Placement
Assemblywoman Julia Brownley

It’s not unusual for foster youth change homes and schools many times during their childhood.  Brownley’s bill would require new school to immediately enroll foster children even if they’re missing their immunization records.

Status:  AB 709 has been ordered to a third reading in the state senate.

Health and Safety

AJR 10: School Based Health Centers
Assemblywoman Julia Brownley

This resolution would declare the Legislature’s support for the school-based health center program, asking Congress to appropriate funds for the program under the 2010 federal health care reform law. The resolution also declares the Legislature’s support for including these centers in the federal Elementary and Secondary Education Act. School based health centers provide health, dental and psychological services targeting the 1.5 million California students without health insurance. Research shows the centers improve academic performance and success by boosting attendance rates.

Status: Awaiting vote on Senate Floor.

SB 614:  Whooping Cough Immunization Grace Period
Senator Christine Kehoe

Kehoe’s bill gives California school districts a 30-day grace period from a new state law that prohibits them from enrolling any student in grades 7 through 12 who hasn’t been vaccinated against whooping cough.

Status: SB 614 passed the senate by a vote of 38-0.  It’s an urgency bill, which means it will take effect immediately if the Governor signs it.

SB 161: Emergency medical assistance: administration of epilepsy medication.
Senator Bob Huff

Since school nurses are becoming a vanishing breed due to budget cuts, this bill would allow teachers or other school personnel to receive medical training to administer a specific drug prescribed to some children with epilepsy.

It’s become a sensitive issue because the medication is a rectal suppository, and school employees are concerned they can be held legally liable if something happens to the child.  Supporters counter that this particular medication must be administered immediately when a child has a seizure and there’s no time to call a parent to come to the school.

Status:  Amended and sent back to the Assembly appropriations committee.

Standards and Assessment

SB 740Pupil Assessment
Senator Loni Hancock

One of the more controversial education bills this session, SB 740 would eliminate second-grade STAR testing.  Hancock points to research warning that high-stakes achievement tests are inappropriate for preschool and early elementary school children, and recommends diagnostic testing instead.

Opponents say that waiting until the end of third grade to learn whether students are working below grade level is too late.

Status:  Scheduled for a vote in the Assembly appropriations committee on August 17.

SB 547: Public School Performance Accountability
Senator Darrell Steinberg

SB 547 would reduce the emphasis on the California Standards Test by limiting the exams to no more than 40 percent of a high school’s overall ranking, and a minimum of 40 percent for middle and elementary schools.

It would also replace the Academic Performance Index (API), with a new system known as the Education Quality Index, or EQI, which would be based on graduation rates and how well schools prepare students for college and career success in addition to test scores. A committee headed by State Superintendent Tom Torlakson would develop other measures.

Status:  Scheduled for a hearing in the Assembly appropriations committee on August 17.

AB 224:  School Accountability:  Academic Performance Index
Assemblywoman Susan Bonilla

This legislation would add some new measures to the state’s Academic Performance Index (API).  Currently, 60 percent of a school’s API ranking comes from students’ scores on the California Standards Tests.

Bonilla’s bill would include other indicators of achievement including graduation rates and preparations for college.

Status: Re-referred to the Senate appropriations committee.