The state Department of Education has received but not yet publicly released IBM’s response to its demand to fix the troubled statewide student data system, known as CALPADS, that the company is building. The problems are serious enough that consultants who did a systems analysis earlier this month concluded that CALPADS was “in critical danger of project/system failure.”
“The system should be stabilized before any further development is conducted,” wrote Sabot Technologies of Folsom in a 15-page report. Even at the risk of further delays to a system that is already a year behind schedule, “the instability and performance problems with the system warrant decisive focused corrective action,” the report said.
Sabot has done several analyses of CALPADS over the past year. In a report in June, Sabot was encouraged by progress that IBM had made. But everything changed in November, when IBM updated software to accommodate the next round of reporting requirements by school districts. The system encountered new instability that, if not fixed, would be compounded when districts file two additional reports later this year.
The problem fit a pattern, Sabot said. “There have been delays in all major releases as a result of a variety of IBM shortcomings. IBM continually causes delays for reasons including quality issues, underestimation of scope, and poor coordination of testing.”
The biggest problem was IBM’s inability to process large quantities of data overnight that districts have uploaded. It’s supposed to be done in eight hours, but IBM had regularly missed the deadline for reasons that aren’t clear. “It is apparent that IBM does not know the root cause of the performance issues,” Sabot said.
IBM is supposed to design a system that will function effectively three years out, which would mean that CALPADS soon should be able to process the overnight data in four hours. IBM has resisted creating tests that would simulate conditions that could be expected under a heavier workload, the report said.
A year ago, Sabot criticized the quality of the work and the experience of the team that IBM assigned to CALPADS. The latest report was equally critical. “Despite these improvements (in communications and coordination), there remain critical issues with quality control in the IBM engineering process. In analyzing the results, the IBM team has demonstrated that they do not possess the technical skills to design and develop the system to contractual specifications.” (How about sending in the team that worked on Watson?)
Noting a “longstanding lack of proactive attention” to problem areas, Sabot recommends that IBM send in a strike team to do a thorough systemwide analysis.
In a letter to IBM earlier this month, Richard Zeiger, the new Chief Deputy Superintendent of Public Instruction, wrote that the Department had determined that IBM’s “substandard work” constituted a material breach of the CALPADS contract.
The Sabot report supported that conclusion: “IBM is grossly out of compliance with their contract schedule and as a result the project is grossly out of compliance with its planned schedule for completion and realization of value as stated in the project approval documents.”
In a statement yesterday, IBM disputed the report’s conclusion and insisted that it remained on track to deliver the completed system by the end of June.
“We disagree with the conclusions of the Sabot Report and it is important to note that CALPADS currently is operational in California school districts who are successfully using it to submit information vital to tracking the progress of the State’s students. IBM continues to work in partnership with the California Department of Education in order to meet the ambitious scope of this project,” it said.
Paul Hefner, director of communications for the Department of Education, declined to speak about IBM’s response to Zeiger’s letter, but he did say that the Deparmtent has noticed “a stepped up level of activity and efforts to improve the performance of the system” by IBM since the letter was written, which the Department views as “hopeful signs at this point.”