Mixed results using iPads

A study conducted in four California school districts found that students studying Algebra I on an iPad did no better overall than students equipped with a traditional textbook.

The results of the 2010-11 study – the largest to date – disappointed Houghton Mifflin Harcourt, the publisher that commissioned the research and had expected better outcomes from the new technology. But at the same time, a company executive  said the firm remains undeterred in developing its digital textbooks and was heartened by scores in Riverside Unified, the one district in the study where students using iPads markedly outperformed their peers. In a white paper that the company published, putting a positive spin on the research, the Riverside teachers in the study extolled the software, which it said motivated students to take charge of  their learning.

HMH Fuse, the software that Houghton Mifflin Harcourt developed exclusively for the iPad, comes equipped with many nifty features: more than 300 video tutorials by the textbook’s author, a scratch pad for note-taking, icons that provide links for support, an ability to record notes by voice, and animated views that walk students through sample problems. Students I spoke with last year at a San Francisco middle school that was part of the study said they liked the features and found them helpful. (HMH has since then improved the note-taking capability, after students complained about its limitations.)

The study found no particular iPad feature directly contributed to math improvement, but collectively they kept  students more engaged; there were also indications – though no hard numbers – that students with iPads did more math at home and after class. The students using the software who did outscore other students tended to have better attitudes, said Denis Newman, president of Empirical Education, the Palo Alto firm that did the research.

Riverside Unified students using Houghten Mifflin Harcourt's Algebra program on an iPad scored 9 percentile points higher than students using the company's Algebra textbooks, an impressive difference. Source: research by Empirical Education, Inc. (Click to enlarge.)
Riverside Unified students using Houghton Mifflin Harcourt's Algebra program on an iPad scored 9 percentile points higher than students using the company's Algebra textbooks, an impressive difference. Source: research by Empirical Education, Inc. (Click to enlarge.)

But overall scores on the California standardized tests and the publisher’s year-end course exam averaged nearly the same for students using iPads and textbooks, after controlling for pretest and demographic differences – except for Riverside, where there was a 9 point increase in the percentile ranking, a significant amount, for those using an iPad. Put another way, by the district’s analysis, 78  percent of students using iPads scored proficient or above on the Algebra CST; 59 percent scored proficient using textbooks.

78 percent of Riverside students using an iPad to learn Algebra I scored proficient or advanced on the state Algebra CST, compared with 59 percent using a standard texbook by the same publisher, Houghton Mifflin Harcourt. Source: research by Empirical Education.  (Click to enlarge.)
78 percent of Riverside students at the Amelia Earhart Middle School using an iPad to learn Algebra I scored proficient or advanced on the state Algebra CST, compared with 59 percent using a standard textbook by the same publisher, Houghton Mifflin Harcourt. Source: Houghton Mifflin. (Click to enlarge.)

Empirical Education did a rigorous analysis, using 11 teachers in six schools across four districts: San Francisco Unified, Fresno Unified, Long Beach Unified, and Riverside. They taught a combined 34 classes ­­– 23 sections with textbooks and 11 sections with iPads, chosen at random. Most of the teachers volunteered to be part of the iPad study, but at least one teacher who proved to be a Luddite was assigned to the study. By the end of the year, nine of the 11 teachers said they would continue to use the iPad if given the choice (sorry, but HMH took them back).

Why the big difference with Riverside? Empirical Education found that the two teachers in Riverside used the iPads as a teaching tool much more extensively than the other nine teachers in the study, and the students used the iPads many more minutes per week in class than all but one of the others – a possible connection.

‘Personalized learning devices’

But Riverside Superintendent Rick Miller has another explanation.  The district has been proactive in deploying mobile technologies among its 42,000 students; it has learned that the best strategy is to encourage students to make iPads and tablets their personalized learning devices, as indispensable as cell phones. Allow students to download their own applications, including music, and they’ll be more prone to access math videos and use the technology for learning. Others districts adopted more restrictive policies, at least at first: They reportedly locked up iPads when they weren’t in use and didn’t allow students to take them home.

The other difference, Miller said, is how the teachers used the iPads. Riverside’s two teachers who volunteered for the study weren’t chosen because they were tech-savvy; they hadn’t owned Apple products. But, Miller  said, they were good math teachers who came to recognize opportunities for differentiating instruction and for assigning  videos at home to introduce concepts.

John Sipe, HMH senior vice president and national sales manager, concurred. “In Riverside, teachers were more comfortable from the beginning. They let the technology organically change the classroom structure and front-load instruction.”

In retrospect, said Sipe said, the company should have done a two-year study, because there was a learning curve to the new technology. And it would spend less time at the start of the year teaching teachers on using the device and more time on classroom strategies with the software.

This year, Riverside has expanded its use of HMH’s iPad Algebra program to four classes in the middle school that participated in the study. However, without the free iPads, students have to supply their own, which has led fewer low-income students to participate. (Parents can buy inexpensive insurance to cover theft or damage.) Meanwhile, Riverside is charging ahead with going digital, experimenting with multiple devices and free and proprietary software in multiple subjects.

HMH Fuse sells for $59 retail; districts can buy it for $49 with a six-year use. It’s also now available for  $19 for a use of one year. Miller said that if  Houghton Mifflin Harcourt releases an app that can run on cheaper Android-based tablets, which will soon sell for under $200, then Riverside would deploy the HMH Fuse program much more extensively. Sipe said an Android app might be available this fall.

Doubts over 8th grade algebra for all

When it comes to flip-flopping, forget the Republican primary and take a look at California’s vacillation on when students should learn algebra. Yesterday, a year and a half after the State Board of Education adopted new math standards, researchers, educators, and policymakers once again sparred over the wisdom of requiring Algebra I for most eighth graders.

When Phil Daro helped write California’s Common Core math standards, he was instructed to base them on evidence, not politics, and to take a close look at math education in the world’s top-performing countries. “What we saw, and what we learned, contradicts a lot of the assumptions on which California mathematics policy is built,” Daro told several hundred people attending Thursday’s Middle Grades Math conference at Stanford University.

The most elemental difference, said Daro, who co-directs UC Berkeley’s Tools for Change, is that even though Algebra I is considered the single most important mathematics subject, California rushes students through it when they’re still in middle school, while high-achieving countries spread it out over three years. “We’re saying let’s spend less time on Algebra I, the most important math; it doesn’t make sense,” Daro said.

8th grade students scoring proficient or better on Algebra I, by race. (Source:  SVEF) Click to enlarge
8th grade students scoring proficient or better on Algebra I, by race. (Source: SVEF) Click to enlarge

Seated at a table in the back of the meeting room, farthest away from the speakers, some of the heaviest hitters in California education glanced at each other and exchanged a quick whisper. Algebra I is a can of worms they’d like to see buried beneath a massive compost pile, preferably in a neighboring state.

It’s been dogging the state at least since the 1997 content standards, which included math standards only through seventh grade. Grades eight and up were organized around content tests, according to a 2011 report commissioned by the Silicon Valley Education Foundation.*   Citing an Education Week article, the report’s authors wrote that “the goal was to increase the number of students enrolled in Algebra I, not to mandate enrollment.”

More than a decade later, under pressure to comply with No Child Left Behind, the State Board of Education made Algebra I mandatory for eighth grade students. That led to a lawsuit, an injunction against the mandate, and flexibility for eighth graders to take Algebra I or an Algebra prep class. Then came Common Core, and California, in a preemptive move, adopted two sets of eighth grade math standards, pre-Algebra for the national standards, and Algebra I for the state.

So it’s understandable if Daro’s recommendation to go more slowly to make sure that students fully comprehend the material – sound policy or not – didn’t elicit any huzzahs from policymakers at the meeting.

The core reason for mandating Algebra I in eighth grade is equity and access for all to college prep courses. Supporters hoped it would stop the practice of tracking low-income and other underserved students away from the A-to-G classes required for admission to the University of California and California State University. But researchers at the meeting warned that the policy, as it’s being implemented, could backfire and make it harder for those students to be successful.

There’s no doubt that it has achieved that goal. According to the SVEF report, the number of students taking Algebra I in eighth grade jumped by 80 percent between 2003 and 2010, with the most dramatic increase among low-income, African American, and Latino students. As that number rises, so too does the number of students reaching proficiency on the Algebra I California Standards Test. Nearly two times as many eighth graders met that bar, according to Algebra Policy in California, published by EdSource.

Then the laws of physics kick in an there’s almost an equal and opposite reaction, with 1.5 times as many of the students scoring below or far below basic. Even though more eighth grade students are taking Algebra I, that doesn’t mean they’ve been equally prepared for it, said Neal Finkelstein, a senior research scientist at WestEd. “There are many patterns of students who are not succeeding early, and are continuing to not succeed later,” said Finkelstein.

Grade 8 algebra enrollment by race. (Source-SVEF) Click to enlarge.EdSource researcher Matt Rosin wanted to know what the chances were of a student who scored basic or below on the seventh grade California Standards test being put in Algebra I. When he analyzed algebra placements and test scores for nearly 70,000 eighth graders during the 2008-09 school year, he found that compared to middle class schools, more students at low-income schools were placed in Algebra I, and more of them scored basic or below on that state test.

“This is the achievement gap in action,” said Rosin. “Schools that have heard the call for greater access to Algebra I are answering the call, but they’re making decisions based on access, not on instruction and support.”

Others would disagree that standards alone are the issue. During a conversation with Daro after the conference, former State Board of Education president Ted Mitchell said teacher preparation is a problem, especially in elementary schools. He would bring in math specialists to help out.

Bruce Arnold seems to share that sentiment. He runs the Mathematics Diagnostic Testing Project at UC San Diego, where teachers learn to identify the specific reasons a student is having difficulty grasping a concept, and get ideas on how to teach that lesson differently.

If students misunderstand any of the prerequisite materials, that will stay with them and trip them up as they move to more advanced classes, explained Arnold. He falls somewhere in the middle when it comes to how much time to spend on the big concepts. Definitely not the three years that schools take in Singapore, however. “I would argue that students should take Algebra I as soon as they are ready,” said Arnold, “and that our goal should be to move students along as fast as they can be moved with success.”

* TOP-Ed is an editorially independent project of the Silicon Valley Education Foundation.

EdSource sees flaw in Algebra for all

A report by the non-partisan research organization EdSource is critical of the distinctly California trend of pushing eighth graders to take Algebra I while acknowledging impressive gains over the past decade in the numbers of low-income, minority children who are mastering the subject.

“A ‘one size fits all’ approach of placing all 8th graders into Algebra I, regardless of their preparation, sets up many students to fail,” concludes “Preparation, Placement, Proficiency: Improving Middle Grades Math Performance.” While 46 percent of eighth graders tested proficient in Algebra I last year (up from 39 percent seven years earlier, even with higher enrollment), 29 percent of students tested far below basic. Many of these students arrived unprepared, and their failure was predictable, given their low scores in the seventh grade math test.

They will be among the 38 percent of students who will repeat Algebra I in ninth grade. The EdSource study didn’t follow what happened to them in ninth grade. But, based on findings by a Noyce Foundation-commissioned study last year, at least half of these students will likely do worse the second time around.

2003: 32% of 8th graders took Algebra I; 39% were proficient
2010: 57% of 8th graders took Algebra I; 46% were proficient

The EdSource report recommends a more nuanced, consistent, districtwide approach to eighth grade math placement. Instead, policies tend vary from school to school. Only a third of districts reported having explicit written placement policies. Superintendents and principals indicated providing a wide access to a rigorous curriculum – equity – was a higher priority for placement than academic appropriateness.

EdSource said that a student’s score on the seventh grade California Standards Test should be a primary, though not sole, factor for placement. ** Based on that, it concluded that “the nearly 40% of 8th graders who scored low basic or lower in grade 7 are clearly not ready for California’s full Algebra I course in 8th grade.”

Many ended up being assigned to Algebra I just the same. They included 27 percent of students who scored far below basic – a failing grade  – on the seventh grade test and  a third of students who scored below basic. 

An unexpected – and fascinating – finding was that students in low-income schools, with parents who are high school graduates, are more likely to be assigned to Algebra I than students from middle-class schools where parents are college educated. And African American and Hispanic eighth graders were more likely to be placed in Algebra I than were white eighth graders with similar preparation. EdSource doesn’t speculate as to why, though advocates have cast universal Algebra I in eighth grade as a civil rights issue as well as the gateway to a four-year university. However, students could still complete the courses required for admission to a UC or CSU campus  after completing Algebra I in ninth grade.

The bigger dilemma is how to place eighth graders who scored high basic or low proficient in seventh grade – and foster their success. Assigning them to General Math is too low a challenge. At the same time, 40 percent of those who scored  low proficient in seventh grade ended up scoring basic on Algebra I the next year.

But these are averages. Some districts clearly do a better job with Algebra among the tweeners – the 30 percent of students in the middle – who, with support and encouragement, can succeed. Based on its previous middle school study, EdSource said that those schools that early on identify students who need extra support show better results.

Another promising option is pre-algebra summer school targeting students who scored high basic or low proficient on the seventh grade test. This summer, between 1,000 and 1,200 students from 13 districts in Santa Clara County will take Stepping Up to Algebra, offered by my employer, Silicon Valley Education Foundation, to prepare them for eighth grade. Unfortunately, every county doesn’t have an education foundation to fund summer schools, which have all but vanished because of state budget cuts.

California’s adoption of the Common Core curriculum presents an opportunity to refine the approach to Algebra, EdSource said. The Common Core curriculum for eighth grade includes a bit of geometry and puts off some of the harder elements of Algebra I, such as quadratic equations, to ninth grade. California, in its adoption, also built those back in to create a dense and overloaded set of standards.

One possibility is two eighth grade sequences: one based on the national Common Core standards, with elements of Algebra. For the 40 percent of students now taking General Math, even this would be a stretch, EdSource said. The other would be similar to the current Algebra I. There would be challenges in creating assessments and the right incentives. But in sorting this out, EdSource said, state leaders should “acknowledge that all students deserve math courses that challenge them, but that all students need not follow an identical path and timeline toward college- and work readiness.”

** Schools don’t received the results from the CST, taken in April, until August – often after the master schedule is finished – a complication for using it for placement. The State Board should demand a quicker turnaround when the contract comes up for renewal next year.