At odds over CALPADS funding

Gov. Jerry Brown has said he would oppose attempts of legislators to put back money for programs he wants cut from the state budget. Funding for two statewide educational databases could become a test of his resolve.

Last week, education finance subcommittees of both the Senate and Assembly Budget Committees restored money for both CALPADS and CALTIDES. Senate President pro Tem Darrell Steinberg is an outspoken advocate for the data systems, which have had support in the Assembly as well, so there’s a good chance the appropriation will be in the  budget the Legislature sends to the governor.

At issue is not a lot of money – about $3.5 million, all of it federal dollars and a fraction of the $10 billion shortfall in the state budget. But it’s become a point of contention with a governor who wants to pare back standardized testing and who questions the value of data in general. Brown proposed cutting funding for both systems in his May revised budget.

CALTIDES, the California Longitudinal Teacher Integrated Data Education System, which would provide insights into teacher preparation programs and effectiveness, is in the conceptual stage. But CALPADS, the California Longitudinal Pupil Achievement Data System, is about a year from completion, which is why Brown’s move surprised even those who have criticized technical and management problems that have delayed the system.

“We acknowledge that CALPADS implementation has been difficult, and the scope too narrow. Delays, technical challenges, and the massive undertaking at the local level needed to collect, maintain, and report data in new ways has been a constant source of frustration, but now is not the time to eliminate CALPADS and CALTIDES,” said the Information Alliance for Education, representing 13 education advocacy groups in a June 2 letter to Brown. The Legislative Analyst’s Office also recommended approval of the funding.

CALPADS, which collects transcript information, attendance numbers, dropout data, and standardized test results of every student in the state for research and federal reporting purposes, was not designed to provide districts and teachers with readily useful information to guide instruction. But, through individual student identification numbers, it will track students as they progress through school and keep their records intact if they change districts. It should be able to identify at-risk students and offer the Legislature insights into which academic programs and supports are working and why.

The case for/against CALPADS

For all of the shortcomings in its implementation, the data system CALPADS has always had plenty of supporters. Gov. Jerry Brown is clearly not among them. He’s proposing to kill funding for it and CALTIDES, a related database yet to be built. I’ve asked five longtime backers to argue why both systems should be saved, and an opponent – a teacher who shares Brown’s skepticism toward standardized tests and statewide databases – to make the case for defunding them. Brown has called for stripping funding for the two systems as part of an overall look at how data can best be used at the local level and whether the state is putting too much energy and attention on standardized tests.

CALPADS (California Longitudinal Pupil Achievement Data System), which came online last fall and is scheduled to be completed next year, collects data on student achievement, enrollment, teacher assignment, and other statewide student information like dropout and graduation rates. CALTIDES (California Longitudinal Teacher Integrated Data Education System) would become the central repository for information on the state’s teacher workforce.

Arun Ramanathan: Kids no longer forgotten

Having kids has forced me to look at the world differently. Back in my twenties, I worked with adults with mental illness and addictions, and with very troubled adolescents. I looked at them and saw people who had made adult choices.

Now that I have kids, I look at people who have “fallen down” and see them as the children they used to be. I imagine the paths they’ve taken and think about the unnecessary sorrows they’ve lived through. Twenty, even 10  years ago, a child could “fall down” academically or in school attendance, and one person might know and be able to help. Today, we are lucky to be able to collect the kind of data that allows us to identify and help the kids who have fallen down and pick them up with targeted supports and interventions.

The problem in California is that this work only happens in isolation in forward-thinking districts and charter schools scattered around the state. As a result, millions of our children fall through the cracks and end up in places like our justice system. With a statewide data system that tracked these students and eventually linked into other government data systems, it wouldn’t have to be this way. We could focus on treating symptoms instead of the resulting diseases.

That’s the promise of CALPADs and CALTIDES. They are the statewide foundation upon which we can build a better future for our children, especially the millions of California’s youth who are low income, highly mobile, or stuck in places such as our foster youth system. Fully funding these systems is an adult decision that we ask our leaders to make on behalf of California’s children.

Arun Ramanathan is executive director of The Education Trust—West, a statewide education advocacy organization.

Joe Simitian: For well-informed choices

Both CALPADS and CALTIDES were created to serve two primary purposes: 1) Report required testing and accountability data to the state and federal governments; and 2) Provide state and local policy makers with accurate longitudinal data in order to meaningfully evaluate education policy and investments.

Longitudinal data, the ability to link data from year to year for each individual student and teacher, is important.  It allows educators and policy makers to see changes over time for individual students, groups of students, or teachers.

In the past, school districts would submit aggregate reports summarizing certain student and teacher statistics for a particular point in time. These aggregate reports provide useful ‘snapshot’ data, but the data from one year’s report often can’t accurately be compared to the data from another year, because each snapshot is of a different cohort. For example, under the current system, when the state compares the achievement of English language learners from one year to the next, it can’t accurately distinguish progress because aggregate data doesn’t distinguish between the old and new sets of students.

Longitudinal data will enable policy makers to compare the achievement from year to year, and more accurately evaluate which programs improve student performance and which ones don’t. Similarly, longitudinal data could also be helpful in evaluating the outcomes of teacher credential programs, or the effectiveness of certain professional development activities.

Policy makers don’t yet have the information needed to make smart, well-informed choices for schools and kids. Taking an extended time-out on funding for CALPADS and CALTIDES won’t help solve the state’s budget problem. The $3.5 million targeted for elimination is one-time federal funding.

It has been a long, slow haul, but California has made significant progress with CALPADS and CALTIDES over the past few years. Now is not the time to stall out or forfeit our gains. If anything, the difficult decisions that the budget crisis presents can only underscore the value of meaningful data to the policy and decision-making processes, and to millions of California students.

State Senator Joe Simitian (D-Palo Alto) is the author SB 1614, which authorized the creation of CALTIDES; SB 1298, which established a process for schools and universities to report data using existing unique statewide student identifiers; and SB 885 (currently pending) to take the next step toward establishing a statewide education data system.

Margaret Gaston: Critical data on our teaching force

In 2001, the Center for the Future of Teaching and Learning and SRI International compiled and distributed data showing that over 42,000 members of California’s teaching workforce lacked even the minimum requirements to teach, and that vastly disproportionate numbers of those underprepared teachers were assigned to low-performing schools serving economically disadvantaged children of color. That information, which documented a severe shortage of fully prepared teachers and pervasive inequities among schools, garnered headlines and drove policy changes to strengthen California’s teacher development system. The revelation of this stark data also challenged the common practice of assigning the least prepared and novice teachers to schools where students arguably need accomplished veteran teachers the most.

Today, policymakers and educators would not have access to that same data. Last year’s veto of funding for the CALPADS system by then-Gov. Arnold Schwarzenegger badly damaged California’s ability to collect and analyze data about the status of teaching and student learning. And now, with the further loss of funding for the CALPADS and CALTIDES data systems, the gap in the availability of reliable information upon which sound policy decisions can be made would be even wider.

We support Gov. Brown’s proposal to reform the state’s data collection systems, making them less bureaucratic and more helpful to school- and district- level decision-making. But we would argue for using what has been developed and paid for thus far as a base for further improvement.

CALPADS is also designed to collect and house critical data on the teacher workforce. That information is essential to developing an adequate pool of teachers with the training, knowledge, and skill necessary to ensure all students are able to reach the high academic goals Californians have set for them. It also reveals the subject matter areas – like math and science – and assignments where teachers are needed most. In times of tight budgets, this data is even more important: Legislators, policymakers, and educators must have the current and accurate information necessary to help them target and leverage limited resources in ways that make every nickel count. Without CALPADS and CALTIDES, California is once again flying blind in its quest to strengthen public education.

Margaret Gaston is president of the Center for the Future of Teaching and Learning.

Rick Miller: Statewide data does matter

Before transitioning to CALPADS and a statewide student identifier system, if a school reported that a student transferred in-state, there was no way to verify the information. But several years ago, after years of infighting, California finally established a unique identifier system and now tracks students between schools and districts. Now, when this data is published, if a district reports that a student transferred to another school and the student doesn’t show up, the student is classified as a dropout on the originating district’s watch. As a result, schools throughout California have tracked down and re-enrolled many of these students. Real-life students have returned to further their education. Statewide data matters

CALPADS is also designed to provide critical information about students: Do they need special education? What courses have they taken? Have they taken the CELDT? With a statewide data system, all of this information can follow the student, saving time and avoiding unnecessary retesting. When the data in CALPADS is linked to CALTIDES, California will be able to measure effectiveness of teacher preparation programs and other educator workforce trends. Ideally, California will eventually have a statewide data system that allows us to track programs across the state so we can judge effectiveness and better broker expertise.

These are all common-sense uses of a statewide data system that the governor’s shortsighted budget proposal puts in jeopardy, without saving one cent for the state, as the system is 100 percent federally funded. If we consign the collection and use of student-level data to districts and schools, we abandon the ability to learn from each other and leverage successful approaches throughout the state. We need information to know what is working well and what is a waste of time and money. In this era of scarcity, using a high-quality data system to inform and foster a continuous learning cycle  – and using federal dollars to do it — just makes sense.

Rick Miller is a principal at Capitol Impact, an education policy advisory firm.

Ted Lempert: Save, fix, and expand it

Gov. Brown is right in asking questions about how CALPADS will support education in California, but wrong in taking an action that will waste millions and undermine progress made thus far.

Ninety-nine percent of districts are successfully using CALPADS to report enrollment. Eliminating funding for the data systems now is the technological equivalent of repealing class-size reduction on the second day of school. Eliminating this funding would create unnecessary turmoil for districts, which would need to retool in the next few months in order to comply with basic federal reporting requirements. It would further undermine basic data functions that CALPADS will fulfill, including:

  • Providing the minimal system needed to effectively manage over $50 billion in educational programs.
  • Efficiently monitoring student mobility, dropout, and records transfer when students move between districts.
  • Ensuring compliance with assurances that California gave when securing $4.9 billion in federal stimulus money.

CALPADS provides efficient uses of data, including automated matches to certify 1.4 million students as eligible for free school meals without further application; it ensures the timely transfer of student records, eliminating delays while identifying at-risk students; it eliminates redundant assessments for English learners as well as those with special needs; and it consolidates several major data collections into one.

Advocates agree that simply meeting No Child Left Behind reporting requirements will never provide the most meaningful benefits that CALPADS can provide. Largely because the original scope of CALPADS has been restricted since its inception, it does not currently provide more robust data linkages, warehousing, dashboards, and reporting needed to fully support state and local needs.

However, CALPADS has finally provided California with the technological equivalent of an advanced operating system; it can readily support more robust data functions when the governor is ready.

Ted Lempert is president of the advocacy group Children Now.

Anthony Cody: Put more faith in teachers, not data systems

We do not need CALPADS. We already have far too much money, time, and energy spent on student performance on tests. The emphasis on these tests has led to a profound distortion of instruction, consuming huge amounts of learning time and vast resources. The bottom line is that these tests are blunt instruments compared to the fine work of a dedicated and intelligent teacher, working in collaboration with peers. The limited information from these tests does not grow in usefulness simply because it is developed in ever finer and more sophisticated detail. Rather, this lends a false air of mathematical certainty to decisions that are much better made by the human beings in direct contact with students.

If not CALPADS, then what?

We have placed far too much faith in data systems, and far too little in the capacity of our teachers and students in responding to the learning challenges they face. California is a huge and diverse state. We can tap the creative potential of our teachers best when we actively engage them in designing curricula and assessments that correspond to the interests and needs of their students. There is so much phony rhetoric about how important and precious teachers are. It is time to give teachers real responsibility – not just for preparing students for tests, but for the complex challenge of life in the 21st century.

An 18-year veteran teacher, Anthony Cody coaches science teachers in Oakland Unified. His blog, Living in Dialogue, is published by Education Week.

CALPADS put on ice

Gov. Jerry Brown is proposing to suspend funding for CALPADS, the state student longitudinal data system, and to stop further planning for CALTIDES, the teacher data base that was to be joined at the hip with CALPADS. That big piece of education news was tucked away in May Revision of the state budget that Brown presented on Monday.

Like his predecessor, Arnold Schwarzenegger, Brown has been critical of CALPADS, which is a year behind schedule. Consultants hired by the Department of Education have sharply criticized state oversight of the system and its builder, IBM. But unlike Schwarzenegger, who believed in CALPADS’ mission, Brown is skeptical of the value of  collecting extensive data on schools and individual students’ achievement and of standardized testing itself. He wants to use the hiatus, he says in a summary of the proposed education budget (page 7), by “carefully reforming testing and accountability requirements to achieve genuine accountability and maximum local autonomy.”

Brown says his administration (presumably through the State Board of Education) will involve parents, scholars, teachers and administrators to develop policies that will reduce the time devoted to standardized testing, eliminate data collecting that’s not useful to local schools, and “restore power to school administrators, teachers and parents” (whatever that means).

Brown’s ambivalence, which he expressed to the State Board during an impromptu visit in January, comes at a time when the State Board must decide how to renew the contract for the California Standardized Tests and whether to join either or both consortiums of  states designing new assessments for the Common Core standards in English language arts and math.

The suspension of work on CALPADS and CALTIDES will eliminate nine jobs funded by about $3.5 million in federal money. CALPADS was established as a way of collecting accountability data required by the federal government. That job will be done by another data system, the California Basic Educational Data System, that existed before CALPADS.

Many advocates of reform had high hopes for the research that CALPADS would provide. Oakland-based Education Trust-West said it was “deeply concerned” about the suspension of funding for the system.

“Without an education data system, it is impossible to make informed decisions on behalf of students as we spend scarce education resources,” the organization said in a statement.  “A step away from CALPADS is a step away from the increased transparency and accountability that is vital to ongoing community support for our public schools.”

Here’s to the data system that works

California can’t get a computer system right. Its K-12 longitudinal data system has been a disaster. Post-secondary institutions are squabbling; a P-20 data system, from preschool to post-college, remains a concept.

You’re heard that before; I’ve written it myself.

But that’s not the complete story. Under the radar, Cal-PASS, not to be confused with CALPADS, is working as designed. By sharing data among school districts, community colleges, and universities and bringing instructors of all levels together, it’s changing classroom practices and, to an extent, policies.

Run on “budget dust,” as a state finance official calls its budget, it’s attracting smart money – about $4 million from foundations (Hewlett, Irvine, Stuart, Gates, Lumina, Walter S. Johnson) – and is expanding. Next week, it will announce the formal creation of the Institute for Evidence-Based Change, a nonprofit at the University of San Diego that will oversee the work of Cal-PASS. Brad Phillips, the Cal-PASS executive director, will head the Institute.

Cal-PASS, the California Partnership for Achieving Student Success, is doing what many mistakenly assumed CALPADS – the California Longitudinal Pupil Achievement Data System – would do: providing data that classroom educators and administrators up and down public education systems can use to answer questions and solve problems. Problems like the duplication of career technical course content in a high school and a local community college, resulting in inefficiency and wasted student effort. Or, in a case  Cal-PASS cites  on its website, the dilemma over why students at a San Diego high school who had passed Algebra had so much trouble with chemistry. The solution was a summer course stressing applied learning, making the connections between algebraic equations and  chemistry formulas. (My colleague, Kathy Baron, recently wrote about a landmark research effort linking Cal-PASS records and the California Child Welfare Database to learn more about the school results of the state’s 62,000 foster care children.)

Cal-PASS has been collecting data for more than a decade and now has 340 million records from 25 million students, including California standardized test results, high school exit exams, and courses students take. Most K-12 districts are part of the system (the largest districts participate), as are all of the state’s community colleges and most of its CSU and UC campuses. The key to its success – and its limitation from a statewide policy  perspective – is that it’s a voluntary collaboration. Institutions sign MOUs determining with whom and how they’ll share data, and it’s regionally focused. About a thousand educators get together monthly  on their own time in 60 regional “professional learning councils” to identify very specific issues to work on.

The encrypted data are accessible to users but not to the public. Teachers can track cohorts of students – how many Hispanics with B’s in 10th grade English in Los Angeles needed remediation in community college – but not individual students.

What makes Cal-PASS user friendly — and is drawing attention from data leaders in other states — are the  internet tools, developed for community colleges and now shared with K-12 teachers, that permit complex searches. That’s why Cal-Pass’ decentralized approach can achieve a lot with a staff  of 20.

By contrast, CALPADS is a one-way street in which districts are mandated to feed records daily to the state.  It was designed as an accountability system for the feds (four-year data on dropout rates are due this summer), not as a local school improvement system for teachers, and it collects far more data, such as attendance records, than Cal-PASS requires of its members. The individual student identifier numbers assigned to every child, enabling the state to track children, the grades they got and teachers who taught them, also raise complex privacy and policy questions.

The ideal statewide data system would combine the capacity and capability of both Cal-PASS and CALPADS. Cal-PASS’s Phillips said that officials from both have begun to discuss how CALPADS data might be made available to Cal-PASS in the future. There’s no point in duplication of collection, he said.

What he didn’t say was that the same higher ed institutions that appear stymied over joint governance of a P-20 data system  are already sharing data, albeit with restrictions, through Cal-PASS.  There is something  to be learned from their experience.

One more stab at P-20 data structure

Call him Senator Sisyphus. With two years before he’s termed out, Sen. Joe Simitian keeps pushing against inertia and bureaucratic resistance to create an effective education database that can track students from preschool to the workplace.

Other states without Silicon Valley savvy have already done so. California committed to establish such a system when it agreed to accept nearly $5 billion in federal stimulus money two years ago. The failure to make progress may have cost the state a chance for Race to the Top money last year.

The problem is unrelated to troubles with CALPADS, the K-12 data system that’s a year behind schedule. The immediate issue is governance: getting all of the disparate parts of California education – the Department of Education, which oversees K-12; CSU; UC; the chancellor’s office of the community college system; the Commission on Teacher Credentialing; and the California Postsecondary Education Commission – to share the data that they’ve already compiled.

Simitian, a Democrat from Palo Alto, has sponsored bills that have moved the process along: one authorizing a database on teachers, which will track education, training, and classroom success; another clearing the way for researchers to do their work. Two years ago, he proposed a governance structure for a P-20 (preschool to workplace) database, but it was changed to a working group, led by the Legislative Analyst’s Office. It didn’t manage to get too far. As Simitian puts it, One of challenges is everyone wants to be in charge but no one wants to be accountable.”

But the working group’s report did recommend creating a joint powers authority, with representatives from the education system, the governor’s office, and the Legislature, that would establish procedures and regulate data sharing. It would hire its own staff and operate independently – and thus potentially override turf wars.

Simitian’s SB 885 would create such a joint powers authority.

There is already precedence for cooperation in the California Partnership for Achieving Student Success (Cal-PASS), in which some K-12 and higher ed institutions have been sharing data for research for more than a decade. But Cal-PASS, while useful, is voluntary.

Simitian says a data system is essential, in asking for additional money for education, to assure taxpayers that dollars are well spent. “We need to make informed choices,” he said, to answer questions like, Did our Head Start investment pay off in college enrollment? Did community college programs lead to employment? Should a school district invest new dollars in teacher training or smaller classes?

There’s no money now for linking the approximately 150 databases that would comprise the P-20 system or for training the people to run it. But creating the governance agreements would put the state in a better position to apply for federal money and move ahead when there are dollars. The state lost out in competition for $250 million in federal data grants two years ago.

The first hearing on SB 885 will be April 27.

Data milestone for foster care

Researchers have linked two California databases in a rare undertaking that holds promise for helping the state’s 62,000 foster youth. The cross-pollination, announced earlier this month at the California Foster Youth Education Summit, connects California’s Child Welfare Database with Cal-PASS, the California Partnership for Achieving Student Success.

Cal-PASS partnered with UC Berkeley’s Center for Social Services Research on the pilot project, which matched students in foster care with those not in the foster system by grade, disability, socioeconomic status, and ethnicity. Researchers say the initial goal was just to see if it’s possible to connect the databases. Once they succeeded at that, they ran some numbers on educational outcomes for foster youth.

“It’s our hope that policy makers will use this to help inform policy development; they seem really hungry for this information,” said Lauren Davis Sosenko, associate director for special projects at Cal-PASS. Although participation in Cal-PASS is voluntary, it has sizable support: Every community college and 75 percent of school districts and public universities supply data. For this first attempt at linking, however, they studied just four counties: Fresno, Sacramento, San Bernardino, and San Diego.

Unexpected Outcomes

Despite the size of the sample, Sosenko said they were able to draw three major implications:

  • Foster youth not only scored lower than the general student population on the California Standards Test (CST), they also did worse when matched for key risk factors that hinder school success, such as disabilities, poverty, and certain ethnicities. In eighth grade math, just 19 percent of foster youth reached proficiency compared with 22 percent of similar students not in foster care and 34 percent of all students tested. This disparity held for all years of high school in English and for grades 9 and 11 in math.
  • Resources can make a world of difference. This may seem obvious on the surface, but the actual numbers are overwhelming. Foster youth in California’s community colleges who received financial aid were 136 percent more likely to remain in school than those who didn’t get money. So, you’d think they’d all be gobbling up the assistance, but that’s not the case. Many weren’t aware that the aid existed, and teachers and counselors, dismissing them as unlikely candidates for college, failed to provide the information. In a finding Sosenko called “criminal,” of the 5,492 foster youth in community college surveyed, only 16 percent received financial aid.  
  • Certain groups of foster youth are more academically vulnerable. Again, it’s the numbers more than the fact of the disparity that makes this so alarming. Disabled foster youth are 75 to 85 percent less likely to achieve proficiency on the California Standards Test than non-disabled foster youth.

Their findings weren’t exactly shocking news. Numerous studies have already shown what’s not working. But by connecting the dots, educators will be able to more accurately target services to specific groups of foster youth at different stages of their education, such as when to start talking about college, and to understand their unique challenges, such as how the number of times a child is moved from home to home affects their success in school.

“I worked with a kiddo yesterday who’s been in 21 schools and he’s 14; it’s hard to put together his educational history,” explained Michelle Lustig, coordinator for foster youth and homeless education in the San Diego County Office of Education.

The county office has had its own copyrighted data-sharing system in place for nearly five years (“September 19, 2006,” as Lustig precisely remembers), known as FYSIS, the Foster Youth Student Information System. Each of the 42 local school districts uploads academic data, which is matched electronically on a daily basis with information from juvenile court, probation, and the Child Welfare System/Case Management System.

The Challenge of Privacy

Access is carefully controlled only to those departments that are legally entitled to it, such as juvenile courts, which are required to monitor the academic process of students in foster care. Judges can pull up a student’s record and see in real time if the child has a lot of absences or is failing classes.

Privacy is a critical issue. The Family Educational Rights and Privacy Act (FERPA) contains strict regulations on who may see a student’s education records. But it’s also interpreted differently by different jurisdictions. The San Diego County Office of Education reads FERPA to say that information can be exchanged to comply with a judicial order or subpoena for health and safety, and for child welfare. However, that excludes data on discipline records like suspensions and expulsions.

FERPA proved more challenging for the Cal-PASS and Child Welfare System project. To make sure the project fell within the law, each student received a unique code that was not linked to either database; those codes were then encrypted before being sent to the researchers. But it proved to be a double-edged sword. On the one hand, no one’s privacy was compromised; on the other hand, there’s no way to check back in a few years to see how the students are faring. That limitation won’t change in the next phase of the project, when Cal-PASS and UC Berkeley’s Center for Social Service Research expand to a statewide and longitudinal study.

Complement Not Compete With CALPADS

The promise of these data systems comes as problems continue with the California Longitudinal Pupil Achievement Data System, the statewide student database known as CALPADS, which is way behind schedule. But neither Sosenko nor Lustig sees it as a competition. CALPADS will have a more robust data set, said Sosenko, because it’s a state-mandated system, and will collect more information from kindergarten through middle school, while Cal-PASS focuses more on the transition from high school to college. Moreover, CALPADS will provide a big-picture look at how foster youth are doing in school, while local systems like San Diego’s can pinpoint which individual students need help.

“I think there’s always been a tension in California around state-level data vs. local data, and at times that feels like it’s been put at odds,” said Lustig. “I don’t think it’s either/or; we need to see across the board how these students are doing rather than just how is Johnny doing, so I think both are great and needed and I hope that it happens.”


Did you know?

  • There are 62,000 foster children and youth in California
  • Three-quarters of foster youth work below grade level in school
  • Nearly 50% drop out of high school
  • One in four foster youth earn a GED instead of a high school diploma
  • More than a third of foster youth who “age out” of the system do not complete a GED or earn a high school diploma
  • Of the 4,000 foster youth who age out of care, 20% will attend college, but only 5% will graduate

Within the first 2 to 4 years after emancipation from foster care:
51% are unemployed
40% are on public assistance
25% become homeless
20% are incarcerated
Source: Child Welfare Dynamic Report System
American Bar Association, Center on Children and the Law
California Alliance of Child and Family Services
California College Pathways Program
California Foster Youth Education Task Force
Child Welfare System/Case Management System
County Welfare Directors Assn. of California Children’s Services
Fostering Media Connections
John Burton Foundation for Children Without Homes
Legal Center for Foster Care and Education (part of ABA)
National Center for Youth Law
The National Center for Homeless Education
UC Berkeley Center for Social Services Research
The Family Educational Rights and Privacy Act (FERPA)

CALPADS ‘in danger of failure’

The state Department of Education has received but not yet publicly released IBM’s response to its demand to fix the troubled statewide student data system, known as CALPADS, that the company is building. The problems are serious enough that consultants who did a systems analysis earlier this month concluded that CALPADS was “in critical danger of project/system failure.”

“The system should be stabilized before any further development is conducted,” wrote Sabot Technologies of Folsom in a 15-page report. Even at the risk of further delays to a system that is already a year behind schedule, “the instability and performance problems with the system warrant decisive focused corrective action,” the report said.

Sabot has done several analyses of CALPADS over the past year. In a report in June, Sabot was encouraged by progress that IBM had made. But everything changed in November, when IBM updated software to accommodate the next round of reporting requirements by school districts. The system encountered new instability that, if not fixed, would be compounded when districts file two additional reports later this year.

The problem fit a pattern, Sabot said. “There have been delays in all major releases as a result of a variety of IBM shortcomings. IBM continually causes delays for reasons including quality issues, underestimation of scope, and poor coordination of testing.”

The biggest problem was IBM’s inability to process large quantities of data overnight that districts have uploaded. It’s supposed to be done in eight hours, but IBM had regularly missed the deadline for reasons that aren’t clear. “It is apparent that IBM does not know the root cause of the performance issues,” Sabot said.

IBM is supposed to design a system that will function effectively three years out, which would mean that CALPADS soon should be able to process the overnight data in four hours. IBM has resisted creating tests that would simulate conditions that could be expected under a heavier workload, the report said.

A year ago, Sabot criticized the quality of the work and the experience of the team that IBM assigned to CALPADS. The latest report was equally critical. “Despite these improvements (in communications and coordination), there remain critical issues with quality control in the IBM engineering process. In analyzing the results, the IBM team has demonstrated that they do not possess the technical skills to design and develop the system to contractual specifications.” (How about sending in the team that worked on Watson?)

Noting a “longstanding lack of proactive attention” to problem areas, Sabot recommends that IBM send in a strike team to do a thorough systemwide analysis.

In a letter to IBM earlier this month, Richard Zeiger, the new Chief Deputy Superintendent of Public Instruction, wrote that the Department had determined that IBM’s “substandard work” constituted a material breach of the CALPADS contract.

The Sabot report supported that conclusion: “IBM is grossly out of compliance with their contract schedule and as a result the project is grossly out of compliance with its planned schedule for completion and realization of value as stated in the project approval documents.”

In a statement yesterday, IBM disputed the report’s conclusion and insisted that it remained on track to deliver the completed system by the end of June.

“We disagree with the conclusions of the Sabot Report and it is important to note that CALPADS currently is operational in California school districts who are successfully using it to submit information vital to tracking the progress of the State’s students. IBM continues to work in partnership with the California Department of Education in order to meet the ambitious scope of this project,” it said.

Paul Hefner, director of communications for the Department of Education, declined to speak about IBM’s response to Zeiger’s letter, but he did say that the Deparmtent has noticed “a stepped up level of activity and efforts to improve the performance of the system” by IBM since the letter was written, which the Department views as “hopeful signs at this point.”

California lags on using data

The release Wednesday of an annual 50-state survey reveals that California is falling behind – way behind – in using statewide student data in effective ways. Despite its ongoing struggle to get CALPADS, the longitudinal student data system, up and built, the primary obstacle is not technology but leadership. If CALPADS were flawless, there still would be this problem: Despite years of talk, the state remains no closer to an agreement on governance: who will manage which data, how it will be shared, who will have access to it, for what purpose, and under what conditions?

California is stuck in the first phase, constructing CALPADS primarily to meet federal reporting requirements, with no will or momentum to take the next step. Other states are moving ahead, positioning to collect, link, and distribute data to help state leaders make smart decisions and to assist teachers and principals with real-time information.

In its sixth annual survey, the non-profit  Data Quality Campaign compared states’ efforts in meeting the “10 essential elements” for a successful statewide data system and 10 actions that states must take toward the ultimate goal of using data effectively to improve student achievement.

In accepting federal stimulus money – $4.9 billion in California’s case – all states agreed to fulfill the 10 elements of a data system by September 2011. And despite its ongoing battle with IBM, CALPADS’ vendor, California is satisfying nine of the 10 elements. They include using a statewide Student Identifier to track students from grade to grade, collecting data on student enrollments, dropouts and transcrips, and matching statewide teacher identifiers with individual students.

But the one missing element, which will put it out of compliance with the feds this fall, is the failure to link K-12 data with higher education data. The connection is critical for school districts to finally learn how well they are preparing students for college and career choices.

The problem is not a lack of data. The higher ed systems have their own extensive databases. But they and other gatekeepers in state government could not come to an agreement, despite more than a year of discussion, on how the data will be linked and who would control it. (Democratic Sen. Joe Simitian, one of the few champions for data, proposed a governance system in SB 1298 in 2008, but it was weakened in the Assembly.)  Forty-one other states have been able to solve this challenge.

No state has taken all 10 actions that the Data Campaign identified as necessary “to create a culture of effective data use.” But Texas, which is developing a rich data system that will inform policy makers on student progress and teachers on individual students, has taken eight actions; 12 states have taken six or seven; and 18 have taken four or five.

The Campaign was charitable in crediting California with two. I’d add asterisks.

The 10 actions are:

  • Link data systems with early learning, postsecondary education, workforce, social services, and other critical agencies so that it’s clear whether students are getting effective services, and schools are preparing students to meet college and career goals;
  • Create stable, sustained support for the data systems;
  • Develop governance structures to define the roles and responsibilities needed to guide data collection and use;
  • Build state data repositories like CALPADS;
  • Implement systems to provide timely access to information while protecting student and teacher privacy;
  • Create progress reports for educators and parents using individual student data to improve student performance;
  • Create reports using longitudinal statistics to guide systemwide improvement efforts. (Decisionmakers and educators would be armed with information about how students progressed from preschool through college.)
  • Develop a P-20/workforce research agenda with universities, researchers, and intermediary groups to explore data for useful information;
  • Promote educator professional development and credentialing to ensure educators know how to access, analyze, and use data appropriately;
  • Promote strategies to raise awareness of available data so that everyone – legislators, parents, researchers – knows how to access and use the information.

California got credit for building a statewide repository and for creating sustained support for the systems. That’s ironic on both counts. The state Department of Education charged IBM this month with doing substandard work on CALPADS and has given it two weeks to fix the problems or default on the contract. The “stable, sustained” support refers to federally funded work by the California School Information Services (CSIS) to maintain CALPADS. In October, the $7 million that Gov. Schwarzenegger vetoed for CALPADS  because he was unhappy with CALPADS delays included CSIS money. Had there not been a technical error in the veto language, the CSIS staff would have been laid off. So much for sustained support.

In its second-round application for the Race to the Top competition, the state and its participating districts laid out a broad post-CALPADS vision for using student data. It envisioned creating a a “one-stop shop” online data portal by 2013, designed to ensure that “the different education datasets, education data collections, and education data reports collected by the State are listed in one central location for ease of use.” It would contain data reports; individual snapshots of schools, districts, and counties; tables presenting comparative data; and downloadable files that can be independently analyzed.

But California not only lost out on Race to the Top but also on the competition for $250 million in data system funding. President Obama is requesting money for state data systems in next year’s budget.

Even when it eventually does perform as designed, CALPADS won’t satisfy legislators’ or educators’ expectations. And California has no money, no data governance structure, no software design and no comprehensive plan to begin to meet them.

Ultimatum to IBM on data system

In an escalation of tensions, the state Department of Education has notified the vendor of CALPADS, the troubled statewide student data system, that it faces a termination of the contract for “substandard” work and a failure to meet deadlines. The Department is giving IBM 15 days to present a plan detailing how it will fix existing problems, avoid future contract violations, and quickly complete the project.

Richard Zeiger, the new Chief Deputy Superintendent of Public Instruction, wrote the bluntly worded letter to an IBM vice president on Feb. 11. He concluded by saying, “CDE reserves all its rights, including, without limitation, the right to reject the corrective action plan at its sole discretion and terminate for default.”

The letter signals that  state Superintendent Tom Torlakson plans to aggressively press IBM to deliver a project that is more than a year late. IBM has been paid half of its $13.9 million contract, Keric Ashley, the director of the department’s Data Management Division, said last month.

Using a unique identifier, the California Longitudinal Pupil Achievement Data System is designed to track every K-12 student through the system, providing data on attendance and courses, and offering researchers insights into teacher training and into programs that work and don’t work in schools. The state is committed to providing the federal government with accurate four-year graduation and dropout rates later this summer – or risk being forced to reimburse the federal government for stimulus money.

But CALPADS has been in turmoil since October, when Gov. Schwarzenegger, frustrated by missed deadlines, vetoed $7 million from the Department’s oversight budget for the project. The project has limped along ever since, but Gov. Jerry Brown, resisting calls to ask the Legislature to restore the money, has called an interagency group to take stock of the project and its overall aims. Brown’s ambivalence toward CALPADS has alarmed CALPADS proponents, who say that California has already fallen behind other states in the use of student data.

The latest skirmish is over Fall 2, the second of four data batches that IBM is to deliver. Zeiger wrote that IBM failed to comply with two demands. An operating manual is now two years late, and IBM  failed to meet the eight-hour window for overnight processing of data – which is  important to districts uploading the information.

Zeiger then offered a general critique. “The overall quality of the services and deliverables provided by IBM under the Contract are substandard and fail to meet the Contract requirements.” He said that logs, district comments, reports, and regular correspondence back up that assessment.

Building up the case for termination, he wrote, “IBM’s continuous failure to meet the Contract Schedule, IBM’s quality failures and IBM’s various failures to satisfy the contract specifications each constitute material breach of the Contract entitling CDE to terminate the Contract, acquire similar services and deliverables and to charge IBM for any excess costs.”

Neither Ashley nor Zeiger would comment on the letter, pending a response from IBM to the 15-day deadline. Update: IBM issued the following statement: “IBM is reviewing the specifics of the February 3rd letter from the California Department of Education and will continue to work in partnership with CDE to meet the ambitious scope of this project. CalPADS currently is operational and California school districts are successfully using it to submit information that is vital to tracking the progress of the State’s students. The project is in its final phase and we are on track to deliver remaining system functionality by the end of this school year.”

Despite veto, no CALPADS layoffs

After a frustrated Gov. Schwarzenegger deleted $6.8 million for the California Longitudinal Pupil Achievement Data System and related operations from the state budget in October, Supt. of Public Instruction Jack O’Connell issued no fewer than five press releases condemning the move and warning of dire consequences. Others were equally critical.

There would be layoffs among those who provide critical CALPADS support, and operations would shut down after Dec. 6, when the veto took effect, O’Connell warned.

“Unless funding is restored quickly for CALPADS, millions of dollars invested in California’s longitudinal education data system will have been wasted, and our state will be at ground zero in collecting student-level data, placing us last among the states in measuring student progress over time,” O’Connell warned in one November press release.

“The Governor’s veto of funding for CALPADS is a travesty,” he said in another.

As it turned out, the sky didn’t fall on CALPADS. No one from the California School Information Services (CSIS), which provided the help desk for districts on CALPADS, was laid off. The reason: The veto message apparently contained an error; the budget line it cited was not for CSIS, but an area of community college spending. Rather than fix the mistake, the governor’s office let it pass.

As for the CALPADS operation, the state Department of Education has shifted money around to keep the project alive for now, said Keric Ashley, the director of the department’s Data Management Division. And the money for IBM, which has the contract to build the software system, was never a target; it was appropriated in a previous year, he said.

That’s not to say the Legislature shouldn’t take action soon to reauthorize the money. A temporary workaround is not a solution, Ashley said. Vacant positions in the CALPADS budget need to be filled to oversee the system’s completion.

Schwarzenegger vetoed the money because of delays and glitches in the system and a lack of confidence that the Department of Education could manage IBM’s contract. He called for the Legislature to demand more accountability.

Brown calls for interagency review

Gov. Brown picked up where Schwarzenegger left off. While not demanding new oversight, his budget calls for (see page 7 of document) withholding money pending an interagency review of CALPADS’  “objectives, usefulness, longer term implications and compatibility with growing federal requirements.” The latter phrase refers to the federal government’s insistence that states create a data system that tracks students from preschool through higher education. CALPADS is only a K-12 data system.

Brown hasn’t yet named the members of the task force.

Meanwhile, work on CALPADS continues, albeit again behind schedule. The system is requiring that districts upload four batches of school and student data this school year. Districts were supposed to complete the first submission, containing enrollment and dropout data, by mid-December. That deadline has now been pushed back to Feb. 4. There remain some glitches with the system and compatibility issues with district software, Ashley said, but the main reason for the delay is to allow budget-strapped and personnel-stressed districts extra time. IBM also started collecting the second batch of information, on courses that students are taking, last month; the deadline for submitting it is March.

The federal government is demanding that states submit a four-year graduation rate, tracking individual students by their ID numbers for the first time, by this fall. Ashley said he was confident the state would comply on time.

Ashley said that IBM has been paid half of its $13.9 million contract; the state will not pay the remainder until it certifies that the remaining data collections are trouble-free, he said.