Mixed results using iPads

A study conducted in four California school districts found that students studying Algebra I on an iPad did no better overall than students equipped with a traditional textbook.

The results of the 2010-11 study – the largest to date – disappointed Houghton Mifflin Harcourt, the publisher that commissioned the research and had expected better outcomes from the new technology. But at the same time, a company executive  said the firm remains undeterred in developing its digital textbooks and was heartened by scores in Riverside Unified, the one district in the study where students using iPads markedly outperformed their peers. In a white paper that the company published, putting a positive spin on the research, the Riverside teachers in the study extolled the software, which it said motivated students to take charge of  their learning.

HMH Fuse, the software that Houghton Mifflin Harcourt developed exclusively for the iPad, comes equipped with many nifty features: more than 300 video tutorials by the textbook’s author, a scratch pad for note-taking, icons that provide links for support, an ability to record notes by voice, and animated views that walk students through sample problems. Students I spoke with last year at a San Francisco middle school that was part of the study said they liked the features and found them helpful. (HMH has since then improved the note-taking capability, after students complained about its limitations.)

The study found no particular iPad feature directly contributed to math improvement, but collectively they kept  students more engaged; there were also indications – though no hard numbers – that students with iPads did more math at home and after class. The students using the software who did outscore other students tended to have better attitudes, said Denis Newman, president of Empirical Education, the Palo Alto firm that did the research.

Riverside Unified students using Houghten Mifflin Harcourt's Algebra program on an iPad scored 9 percentile points higher than students using the company's Algebra textbooks, an impressive difference. Source: research by Empirical Education, Inc. (Click to enlarge.)
Riverside Unified students using Houghton Mifflin Harcourt's Algebra program on an iPad scored 9 percentile points higher than students using the company's Algebra textbooks, an impressive difference. Source: research by Empirical Education, Inc. (Click to enlarge.)

But overall scores on the California standardized tests and the publisher’s year-end course exam averaged nearly the same for students using iPads and textbooks, after controlling for pretest and demographic differences – except for Riverside, where there was a 9 point increase in the percentile ranking, a significant amount, for those using an iPad. Put another way, by the district’s analysis, 78  percent of students using iPads scored proficient or above on the Algebra CST; 59 percent scored proficient using textbooks.

78 percent of Riverside students using an iPad to learn Algebra I scored proficient or advanced on the state Algebra CST, compared with 59 percent using a standard texbook by the same publisher, Houghton Mifflin Harcourt. Source: research by Empirical Education.  (Click to enlarge.)
78 percent of Riverside students at the Amelia Earhart Middle School using an iPad to learn Algebra I scored proficient or advanced on the state Algebra CST, compared with 59 percent using a standard textbook by the same publisher, Houghton Mifflin Harcourt. Source: Houghton Mifflin. (Click to enlarge.)

Empirical Education did a rigorous analysis, using 11 teachers in six schools across four districts: San Francisco Unified, Fresno Unified, Long Beach Unified, and Riverside. They taught a combined 34 classes ­­– 23 sections with textbooks and 11 sections with iPads, chosen at random. Most of the teachers volunteered to be part of the iPad study, but at least one teacher who proved to be a Luddite was assigned to the study. By the end of the year, nine of the 11 teachers said they would continue to use the iPad if given the choice (sorry, but HMH took them back).

Why the big difference with Riverside? Empirical Education found that the two teachers in Riverside used the iPads as a teaching tool much more extensively than the other nine teachers in the study, and the students used the iPads many more minutes per week in class than all but one of the others – a possible connection.

‘Personalized learning devices’

But Riverside Superintendent Rick Miller has another explanation.  The district has been proactive in deploying mobile technologies among its 42,000 students; it has learned that the best strategy is to encourage students to make iPads and tablets their personalized learning devices, as indispensable as cell phones. Allow students to download their own applications, including music, and they’ll be more prone to access math videos and use the technology for learning. Others districts adopted more restrictive policies, at least at first: They reportedly locked up iPads when they weren’t in use and didn’t allow students to take them home.

The other difference, Miller said, is how the teachers used the iPads. Riverside’s two teachers who volunteered for the study weren’t chosen because they were tech-savvy; they hadn’t owned Apple products. But, Miller  said, they were good math teachers who came to recognize opportunities for differentiating instruction and for assigning  videos at home to introduce concepts.

John Sipe, HMH senior vice president and national sales manager, concurred. “In Riverside, teachers were more comfortable from the beginning. They let the technology organically change the classroom structure and front-load instruction.”

In retrospect, said Sipe said, the company should have done a two-year study, because there was a learning curve to the new technology. And it would spend less time at the start of the year teaching teachers on using the device and more time on classroom strategies with the software.

This year, Riverside has expanded its use of HMH’s iPad Algebra program to four classes in the middle school that participated in the study. However, without the free iPads, students have to supply their own, which has led fewer low-income students to participate. (Parents can buy inexpensive insurance to cover theft or damage.) Meanwhile, Riverside is charging ahead with going digital, experimenting with multiple devices and free and proprietary software in multiple subjects.

HMH Fuse sells for $59 retail; districts can buy it for $49 with a six-year use. It’s also now available for  $19 for a use of one year. Miller said that if  Houghton Mifflin Harcourt releases an app that can run on cheaper Android-based tablets, which will soon sell for under $200, then Riverside would deploy the HMH Fuse program much more extensively. Sipe said an Android app might be available this fall.

The case for/against CALPADS

For all of the shortcomings in its implementation, the data system CALPADS has always had plenty of supporters. Gov. Jerry Brown is clearly not among them. He’s proposing to kill funding for it and CALTIDES, a related database yet to be built. I’ve asked five longtime backers to argue why both systems should be saved, and an opponent – a teacher who shares Brown’s skepticism toward standardized tests and statewide databases – to make the case for defunding them. Brown has called for stripping funding for the two systems as part of an overall look at how data can best be used at the local level and whether the state is putting too much energy and attention on standardized tests.

CALPADS (California Longitudinal Pupil Achievement Data System), which came online last fall and is scheduled to be completed next year, collects data on student achievement, enrollment, teacher assignment, and other statewide student information like dropout and graduation rates. CALTIDES (California Longitudinal Teacher Integrated Data Education System) would become the central repository for information on the state’s teacher workforce.

Arun Ramanathan: Kids no longer forgotten

Having kids has forced me to look at the world differently. Back in my twenties, I worked with adults with mental illness and addictions, and with very troubled adolescents. I looked at them and saw people who had made adult choices.

Now that I have kids, I look at people who have “fallen down” and see them as the children they used to be. I imagine the paths they’ve taken and think about the unnecessary sorrows they’ve lived through. Twenty, even 10  years ago, a child could “fall down” academically or in school attendance, and one person might know and be able to help. Today, we are lucky to be able to collect the kind of data that allows us to identify and help the kids who have fallen down and pick them up with targeted supports and interventions.

The problem in California is that this work only happens in isolation in forward-thinking districts and charter schools scattered around the state. As a result, millions of our children fall through the cracks and end up in places like our justice system. With a statewide data system that tracked these students and eventually linked into other government data systems, it wouldn’t have to be this way. We could focus on treating symptoms instead of the resulting diseases.

That’s the promise of CALPADs and CALTIDES. They are the statewide foundation upon which we can build a better future for our children, especially the millions of California’s youth who are low income, highly mobile, or stuck in places such as our foster youth system. Fully funding these systems is an adult decision that we ask our leaders to make on behalf of California’s children.

Arun Ramanathan is executive director of The Education Trust—West, a statewide education advocacy organization.

Joe Simitian: For well-informed choices

Both CALPADS and CALTIDES were created to serve two primary purposes: 1) Report required testing and accountability data to the state and federal governments; and 2) Provide state and local policy makers with accurate longitudinal data in order to meaningfully evaluate education policy and investments.

Longitudinal data, the ability to link data from year to year for each individual student and teacher, is important.  It allows educators and policy makers to see changes over time for individual students, groups of students, or teachers.

In the past, school districts would submit aggregate reports summarizing certain student and teacher statistics for a particular point in time. These aggregate reports provide useful ‘snapshot’ data, but the data from one year’s report often can’t accurately be compared to the data from another year, because each snapshot is of a different cohort. For example, under the current system, when the state compares the achievement of English language learners from one year to the next, it can’t accurately distinguish progress because aggregate data doesn’t distinguish between the old and new sets of students.

Longitudinal data will enable policy makers to compare the achievement from year to year, and more accurately evaluate which programs improve student performance and which ones don’t. Similarly, longitudinal data could also be helpful in evaluating the outcomes of teacher credential programs, or the effectiveness of certain professional development activities.

Policy makers don’t yet have the information needed to make smart, well-informed choices for schools and kids. Taking an extended time-out on funding for CALPADS and CALTIDES won’t help solve the state’s budget problem. The $3.5 million targeted for elimination is one-time federal funding.

It has been a long, slow haul, but California has made significant progress with CALPADS and CALTIDES over the past few years. Now is not the time to stall out or forfeit our gains. If anything, the difficult decisions that the budget crisis presents can only underscore the value of meaningful data to the policy and decision-making processes, and to millions of California students.

State Senator Joe Simitian (D-Palo Alto) is the author SB 1614, which authorized the creation of CALTIDES; SB 1298, which established a process for schools and universities to report data using existing unique statewide student identifiers; and SB 885 (currently pending) to take the next step toward establishing a statewide education data system.

Margaret Gaston: Critical data on our teaching force

In 2001, the Center for the Future of Teaching and Learning and SRI International compiled and distributed data showing that over 42,000 members of California’s teaching workforce lacked even the minimum requirements to teach, and that vastly disproportionate numbers of those underprepared teachers were assigned to low-performing schools serving economically disadvantaged children of color. That information, which documented a severe shortage of fully prepared teachers and pervasive inequities among schools, garnered headlines and drove policy changes to strengthen California’s teacher development system. The revelation of this stark data also challenged the common practice of assigning the least prepared and novice teachers to schools where students arguably need accomplished veteran teachers the most.

Today, policymakers and educators would not have access to that same data. Last year’s veto of funding for the CALPADS system by then-Gov. Arnold Schwarzenegger badly damaged California’s ability to collect and analyze data about the status of teaching and student learning. And now, with the further loss of funding for the CALPADS and CALTIDES data systems, the gap in the availability of reliable information upon which sound policy decisions can be made would be even wider.

We support Gov. Brown’s proposal to reform the state’s data collection systems, making them less bureaucratic and more helpful to school- and district- level decision-making. But we would argue for using what has been developed and paid for thus far as a base for further improvement.

CALPADS is also designed to collect and house critical data on the teacher workforce. That information is essential to developing an adequate pool of teachers with the training, knowledge, and skill necessary to ensure all students are able to reach the high academic goals Californians have set for them. It also reveals the subject matter areas – like math and science – and assignments where teachers are needed most. In times of tight budgets, this data is even more important: Legislators, policymakers, and educators must have the current and accurate information necessary to help them target and leverage limited resources in ways that make every nickel count. Without CALPADS and CALTIDES, California is once again flying blind in its quest to strengthen public education.

Margaret Gaston is president of the Center for the Future of Teaching and Learning.

Rick Miller: Statewide data does matter

Before transitioning to CALPADS and a statewide student identifier system, if a school reported that a student transferred in-state, there was no way to verify the information. But several years ago, after years of infighting, California finally established a unique identifier system and now tracks students between schools and districts. Now, when this data is published, if a district reports that a student transferred to another school and the student doesn’t show up, the student is classified as a dropout on the originating district’s watch. As a result, schools throughout California have tracked down and re-enrolled many of these students. Real-life students have returned to further their education. Statewide data matters

CALPADS is also designed to provide critical information about students: Do they need special education? What courses have they taken? Have they taken the CELDT? With a statewide data system, all of this information can follow the student, saving time and avoiding unnecessary retesting. When the data in CALPADS is linked to CALTIDES, California will be able to measure effectiveness of teacher preparation programs and other educator workforce trends. Ideally, California will eventually have a statewide data system that allows us to track programs across the state so we can judge effectiveness and better broker expertise.

These are all common-sense uses of a statewide data system that the governor’s shortsighted budget proposal puts in jeopardy, without saving one cent for the state, as the system is 100 percent federally funded. If we consign the collection and use of student-level data to districts and schools, we abandon the ability to learn from each other and leverage successful approaches throughout the state. We need information to know what is working well and what is a waste of time and money. In this era of scarcity, using a high-quality data system to inform and foster a continuous learning cycle  – and using federal dollars to do it — just makes sense.

Rick Miller is a principal at Capitol Impact, an education policy advisory firm.

Ted Lempert: Save, fix, and expand it

Gov. Brown is right in asking questions about how CALPADS will support education in California, but wrong in taking an action that will waste millions and undermine progress made thus far.

Ninety-nine percent of districts are successfully using CALPADS to report enrollment. Eliminating funding for the data systems now is the technological equivalent of repealing class-size reduction on the second day of school. Eliminating this funding would create unnecessary turmoil for districts, which would need to retool in the next few months in order to comply with basic federal reporting requirements. It would further undermine basic data functions that CALPADS will fulfill, including:

  • Providing the minimal system needed to effectively manage over $50 billion in educational programs.
  • Efficiently monitoring student mobility, dropout, and records transfer when students move between districts.
  • Ensuring compliance with assurances that California gave when securing $4.9 billion in federal stimulus money.

CALPADS provides efficient uses of data, including automated matches to certify 1.4 million students as eligible for free school meals without further application; it ensures the timely transfer of student records, eliminating delays while identifying at-risk students; it eliminates redundant assessments for English learners as well as those with special needs; and it consolidates several major data collections into one.

Advocates agree that simply meeting No Child Left Behind reporting requirements will never provide the most meaningful benefits that CALPADS can provide. Largely because the original scope of CALPADS has been restricted since its inception, it does not currently provide more robust data linkages, warehousing, dashboards, and reporting needed to fully support state and local needs.

However, CALPADS has finally provided California with the technological equivalent of an advanced operating system; it can readily support more robust data functions when the governor is ready.

Ted Lempert is president of the advocacy group Children Now.

Anthony Cody: Put more faith in teachers, not data systems

We do not need CALPADS. We already have far too much money, time, and energy spent on student performance on tests. The emphasis on these tests has led to a profound distortion of instruction, consuming huge amounts of learning time and vast resources. The bottom line is that these tests are blunt instruments compared to the fine work of a dedicated and intelligent teacher, working in collaboration with peers. The limited information from these tests does not grow in usefulness simply because it is developed in ever finer and more sophisticated detail. Rather, this lends a false air of mathematical certainty to decisions that are much better made by the human beings in direct contact with students.

If not CALPADS, then what?

We have placed far too much faith in data systems, and far too little in the capacity of our teachers and students in responding to the learning challenges they face. California is a huge and diverse state. We can tap the creative potential of our teachers best when we actively engage them in designing curricula and assessments that correspond to the interests and needs of their students. There is so much phony rhetoric about how important and precious teachers are. It is time to give teachers real responsibility – not just for preparing students for tests, but for the complex challenge of life in the 21st century.

An 18-year veteran teacher, Anthony Cody coaches science teachers in Oakland Unified. His blog, Living in Dialogue, is published by Education Week.