DreamBox aces test: study shows edutainment program boosts exam scores

Maybe games can teach kids something after all.

A study released today by SRI International shows that DreamBox Learning, the education startup that Netflix CEO Reed Hastings bought last year, can be linked to significant gains in standardized test scores. The findings will likely spur discussion about the success of “hybrid” elementary schools, where computers do much of the teaching, as well as the increasing impact of Silicon Valley investment on the public education system.

DreamBox has created math software that is “adaptive.” That is, a child can log into the online game and solve problems and the game can react to the skill level of the child. If the child does well, the game adapts the lessons so that they are harder. If the child needs more help, the game adapts the lessons so they are easier.

The study was conducted over a 16-week period in the fall of 2010 at Rocketship Education, a K-5 charter school network, also backed by Reed Hastings, that serves inner-city youth in San Jose, Calif. Students at Rocketship schools spend at least 25 percent of their academic day using instructional software like DreamBox, and the rest in direct teacher-led instruction.

Rocketship administered the Northwest Evaluation Association’s (NWEA) test, a commonly used standardized mathematics test, at the beginning of the study in September 2010, and again after the study in January and February of 2011. A total of 583 K-1 students from Rocketship’s three schools in San Jose participated. Students were randomly assigned to a treatment and control group.  All students in the study received 100-110 minutes per day of math instruction in the traditional classroom in addition to varying amounts of time with DreamBox.

The study found that students who received additional online math instruction through DreamBox—22 hours over the 16-week period compared to 5 hours in the control group—scored an average of 2.3 points higher on the NWEA math test. For the average student, that would be equivalent to progressing 5.5 points in percentile ranking (for example, from 50 percent to 55.5 percent) in 16 weeks. As educational interventions go, that’s a big leap in a short time period, and it actually reduces costs, since no credentialed teacher is needed to supervise.

On a large scale, many education reformers think this kind of technology could help solve the financial and performance problems in public education. Others fear that attempts to replace teacher-led instruction with computers could result in students losing out on important social skills.

Rocketship math teacher and soon-to-be principal Andrew Elliott-Chandler says there’s no need for teachers to fear being replaced, although their roles will shift. “A lot of the grunt work of teaching is switched out for getting to know kids,” he says. “That’s one of the exciting changes. As learning the math facts shifts to the Learning Lab, there is a whole array of activities and programming that I no longer have to do in the classroom and instead, I’m able to spend more time looking at my kids’ needs… really exploring more high-engagement, high-leverage lessons with my kids.”

Rocketship’s Chief Schools Officer Aylon Samouha says the organization is planning further studies internally to determine, for example, which standards the available software programs like DreamBox teach effectively, and how they compare with additional tutoring and class time for teaching given concepts.

Both Samouha and Elliot-Chandler say the findings instill confidence in the use of technology to deliver instruction. “It helps me put my trust in the Learning Lab in a more absolute way,” Elliot-Chandler said. “I felt confident before that their time was well-spent there, and it’s nice to have the proof.”