Information for Reading Program Developers

Reading Comprehension: Testing What Works

Mathematica and RMC Research Corporation are conducting an important new study for the U.S. Department of Education to learn how to improve reading comprehension in science and social studies for fifth graders. Researchers and partnerships of researchers, publishers, and organizations involved in professional development for instruction in reading comprehension were invited to apply to participate in the two-year impact evaluation. This study will produce scientifically based evidence of their programs' effectiveness.

Through a Request for Proposals (RFP) released in January 2005 and again in May 2005, Mathematica and RMC Research Corporation selected five supplementary programs. Each program is being implemented in about 20 schools across 8 to 10 school districts around the country during the 2006–2007 school year. The study targets schools serving students from low-income families, though some schools may serve students from families with average or above average incomes. Before implementation, each program developer pilot tested and refined their supplementary programs during the 2005–2006 school year in three schools and demonstrated their capacity to provide professional development and follow-up support to ensure quality implementation. Each program is working with about 60 teachers.

There was a three-stage process for review of proposals. During the first stage, RMC reviewed all proposals for minimum qualifications. In the second stage, proposals that met the minimum qualifications were forwarded to a panel of experts for review and evaluation. The third stage consisted of in-person interviews of finalists.

The panel of experts evaluated proposals using the following criteria:

• Theoretical and empirical support for the supplementary program approach
• Evidence of the specific supplementary program’s efficacy or effectiveness
• Quality of the supplementary program design, including the pedagogical skills and techniques, intensity and quality of teacher training and follow-up support, quality of training and support materials and strategies, and quality of classroom activity materials
• Demonstrated capacity to implement the supplementary program in multiple and geographically diverse sites
• Appropriateness of the supplementary program for use with expository text for students in grade 5
• Reasonableness of proposed costs of the supplementary program materials, training, and follow-up support

Finalists were invited to the Washington, DC, area to present an overview of their proposal and respond to questions from the expert panel. The panel then recommended supplementary programs for study. Following approval by the U.S. Department of Education’s Institute for Education Sciences, contracts were negotiated with each of the selected developers. The period of performance began in August 2005 and will continue through the 2006–2007 school year (approximately two years).

Stephen L. Murray, Co-Principal Investigator
RMC Research Corporation
522 SW Fifth Avenue, Suite 1407
Portland, OR 97204-2131
Phone 503-223-8248