Forum for Education and Democracy on "Race to the Top"

http://forumforeducation.org/node/491

Letter to Secretary Duncan

August 27, 2009 — George Wood

Office of Secondary and Elementary Education
c/o Race to the Top Fund
U.S. Department of Education
400 Maryland Avenue, SW
Room 3W329
Washington, DC 20202

To Whom It May Concern:

We are writing to submit our comments regarding the U.S. Department of Education’s (“Department”) notice of proposed priorities, requirements, definitions, and selection criteria pertaining to the Race to the Top Fund (“Fund”). These comments reflect the Forum for Education and Democracy’s (“Forum”) concerns as well as our recommendations for how to improve the proposed regulations.

Like the No Child Left Behind Act before it, the Race to the Top Fund appears to reinforce an overreliance on traditional standardized tests as a measure of student achievement. While standardized tests have their place in the educational system, they are most appropriately used when they are considered as one measure among a balanced scorecard of quantitative and qualitative measures, including performance assessments, that provide more accurate and holistic indicators of a student’s growth across content areas and skill levels. The research base documenting the limitations of large-scale, multiple-choice standardized tests as the sole measure of student performance is substantial.¹ Such assessments, when used for high-stakes purposes, have been found to narrow the curriculum to the subjects, skills, and formats demanded by the tests, and to reduce the cognitive demands of classroom instruction – including the use of extended written products, research, scientific investigations, and complex problem solving that develop the skills students need in a 21st century economy. When tied to school accountability, the use of high-stakes tests has also been found to increase rates of student drop out and push out. It would therefore be inappropriate for the Department to ignore this evidence in its efforts to improve the conditions for student academic achievement.

It would be equally misguided to rely solely on the results of students’ standardized tests for hiring, retention, and tenure decisions for teachers and principals. Such a move would create even more intense incentives to teach to currently narrow tests while creating disincentives for teachers and schools to serve the neediest students. Although efforts are underway to explore the use of “value-added modeling” (VAM) to tie student score gains to individual teachers, most researchers agree that these methods are too unstable and too vulnerable to many sources of error, and to other student and school factors, to be used for teacher evaluation. For example, a major report by the RAND Corporation concluded that:

The research base is currently insufficient to support the use of VAM for high-stakes decisions about individual teachers or schools.²

Similarly, Henry Braun of the Educational Testing Service concluded in his review of research:

VAM results should not serve as the sole or principal basis for making consequential decisions about teachers. There are many pitfalls to making causal attributions of teacher effectiveness on the basis of the kinds of data available from typical school districts. We still lack sufficient understanding of how seriously the different technical problems threaten the validity of such interpretations.³

Consistent with the views expressed above, the Forum recommends that the Department incorporate the following changes:

• Eliminate the requirement that student achievement be measured exclusively by state assessments for tested grades and subjects: The definition of student growth laudably mentions that student achievement data “may be measured by a variety of approaches.” It then goes on to say that “any approach must be statistically rigorous and based on student achievement (as defined in this notice).” Student achievement is then defined as “a student’s score on the State assessment” for tested grades and subjects. The definition does not allow for alternative measures of student learning and performance for tested grades and subjects. To correct this inconsistency, and to acknowledge the substantial research base noted above, the language for the student achievement definition for tested grades and subjects (subsection a) should read, “A student’s performance on a variety of measures that may include, but is not limited to, a student’s score on the State’s assessment under section 1111(b)(3) of the ESEA.”
• Explicitly include a range of performance assessments and other evidence about student and school factors in longitudinal data systems: In furtherance of the laudable goal of supporting the use of a variety of performance-based measurements that can more accurately demonstrate student achievement, it would be appropriate for the Department to recognize explicitly the importance of ensuring consideration of different types of assessments, including state and local performance-based measures, in data and reporting systems. The Forum recommends that States and LEAs be allowed to report and equally weight local performance assessments alongside current State tests. Similarly, State longitudinal data systems should have the capacity to include data from these assessments
• Eliminate Race to the Top’s emphasis on the use of State assessments as a tool for high school exit decisions. In Section (A) (3), sub-section (b) of the Race to the Top guidance, the Department should replace language that reads: “Aligning high school exit criteria” with the proposed new internationally benchmarked assessments with language that seeks alignment of “high school curriculum expectations” with these assessments. Around the world, the intellectually ambitious assessments that are a part of high-achieving nations’ high school programs are used to guide curriculum, inform course grades and college admissions, and support instructional improvement, but they are not used to deny diplomas to students.⁴ This allows the assessments to represent much higher standards while avoiding the often dysfunctional consequences of exit exams.⁵
• Revise the requirement that teachers and principals be evaluated based on state standardized test scores. Efforts to incorporate evidence of student learning in evaluations of teachers and principals personnel should ultimately consider a balanced set of quantitative and qualitative data that reflect teacher practices and performance in the classroom, as well as a broad range of contributions to student learning. Given that strong research cautions against the primary use of standardized test scores to draw inferences about teacher effectiveness, the Department should encourage states to develop and use performance assessments of teachers that can reliably and validly assess the use of teaching practices known to be associated with student achievement gains and to experiment with a range of strategies to incorporate evidence of student learning and accomplishment into teacher and principal evaluation tools.
  

While testing can be useful to illuminate the status of different groups in terms of achievement – and would be more so if the measures used were more robust and thoughtful – it has not reduced inequalities in opportunities to learn that underlie the achievement gap. A large body of research demonstrates that children in low-performing public schools are disadvantaged by a lack of monetary and material resources.⁶ Despite clear evidence pointing to the academic consequences of these inequities, the Department’s Race to the Top guidance fails to ask States to close these resource disparities as a method for improving school-level conditions that can help raise student achievement and close performance gaps in struggling schools. Indeed, with regard to overall funding, Race to the Top only requires States to demonstrate that their 2009 funding allocation for public education is on par with or greater than their 2008 levels. In recent months, the Department of Education has not required states even to maintain equalization plans ordered by the courts. This failure represents a huge missed opportunity to incentivize states to close the funding disparities facing resource-poor public schools—many of which have been, or are at risk of being, labeled failing.

Ironically, the Race to the Top guidance recognizes the importance of funding equity for increasing the supply of high quality charter schools (Section (D)(2), sub-sections (iii) and (vi)), while ignoring its importance for increasing the supply of high quality schools in the rest of the public education sector. Not only does this fail to address historically State-sanctioned funding inequities across schools and districts, by recognizing the importance of funding for charters and not for public schools, the guidance also promotes an unfair funding bias against struggling public schools that need additional resources to improve. As an organization that supports all public schools – charter or otherwise – the Forum would find fault with a funding bias in either direction.

The Forum is pleased that States are required to prioritize high-need LEAs when distributing the 50 percent of Race to the Top funds designated for LEAs. The Forum is also pleased that Race to the Top requires States to submit a plan for equalizing the distribution of teachers—particularly in low-resource schools. However, the guidance requiring expansion of alternative certification programs is as likely to undermine this goal as to help achieve it. While some alternative route programs have effectively trained teachers for high-need schools and kept them in the classroom, others have offered too little preparation and have produced teachers who are both significantly less effective and more likely to leave teaching quickly. This has contributed to the churn in poor urban and rural schools and exacerbated the supply problems they face.⁷

Highly-effective teachers are an essential element for student academic achievement. Research shows that students in low-resource schools do not have access to these teachers at the same rate as students in high-resource schools. It is therefore vital that States not only encourage equal distribution of highly-effective teachers, but also invest in high-quality preparation for these teachers so they can be successful in any school or classroom setting.

This preparation would be improved by attention to research that has documented the features of teacher education programs that produced greater effectiveness among their graduates in supporting reading and mathematics gains for students. These programs:

• Have well-supervised student teaching experiences that are also well-matched to the subjects, grade levels, and students they would later teach;
• Have more coursework in reading and mathematics content and teaching methods;
• Focus their courses on helping candidates acquire specific practices and tools that they then apply in their student teaching or practicum experiences;
• Enable candidates to study the specific curricular materials they will teach;
• Require a performance assessment or portfolio of their work done in classrooms with students, demonstrating their capacity to plan, teach, and assess students effectively.⁸

Finally, preparation offered by both traditional and alternative programs can be assessed and improved by requiring the use of a teacher performance assessment to evaluate teachers’ abilities actually to teach as a condition of initial licensure and as a condition of accreditation for programs, as California has done and as a set of nearly 20 other states, working under the auspices of the Council of Chief State School Officers, are now considering.

Consistent with the observations outlined above, the Forum recommends the following:

• Require states to submit high-quality plans for closing school and district funding disparities as a condition for receiving a Race to the Top grant. It is inappropriate to ignore the role that inequities in school and district funding play in reinforcing disparities in educational outcomes. Race to the Top represents leverage that can be used to encourage states to close historical gaps in funding that disproportionately affect schools and districts serving high-poverty areas.
  
• Include State Reform Conditions Criteria that emphasize the importance of funding equity for public schools. Include language in Section (D)(3) that states: “(i) The extent to which the State’s lowest-achieving or persistently lowest performing schools receive equitable funding, compared to its highest-achieving or persistently highest performing schools, and a commensurate share of local, State, and Federal program and revenue sources.” And, “(ii) The extent to which the State provides its lowest-achieving or persistently lowest performing schools with material and facilities funding (for purchasing supplies and materials and for leasing, purchasing, or making improvements to facilities), assistance with material and facilities acquisition, and the ability to share in bonds or other levies benefiting the State’s highest-achieving or persistently highest performing schools.”
  
• Redefine persistently lowest-performing schools to include measures that recognize historic funding disparities and the importance of integrating a balanced scorecard of assessments. Revised language should read, “When considering which schools are the lowest-achieving, the State must consider the ratio at which the lowest-performing schools are under-resourced as compared to the State’s persistently highest-performing schools, taking cost of education differentials associated with location and pupil needs into account, the length of time for which the lowest-performing schools are under-resourced as compared to the State’s persistently highest-performing schools, and the absolute performance of schools on the State’s assessments in reading/language arts and mathematics alongside other quantitative and qualitative assessments that demonstrate school progress.”
  
• Replace the requirement that states expand alternate routes to teacher certification with a requirement that States provide high-quality preparation for teachers regardless of their path to certification. This requirement should call for states to evaluate both their traditional and alternate route programs and ensure that they include the features of programs found to increase teacher effectiveness, as well as produce teachers who are able to demonstrate, in a rigorous, valid, and reliable performance assessment, that they are prepared to teach competently from their first day in the classroom as teacher of record. States should be required to expand preparation options that are highly successful at producing effective teachers. While teachers may take alternate paths to certification and the classroom, all teachers are clearly in need of high-quality preparation that allows them to effectively advance student learning from the very first day they step into a classroom.

Finally, we wish to underscore the myriad well-publicized data on student dropouts which show that males, students from racial and ethnic groups, and students from low-resource schools have higher rates of leaving school before receiving their high school diploma.⁹ There has been much discussion about NCLB’s “push-out effect”—a phenomenon in which poor performing students are encouraged to leave school in order to improve schools’ Annual Yearly Progress performance. The Forum is deeply concerned that the Race to the Top guidance contains language that may unintentionally accelerate, rather than reverse, the push-out effect.

Specifically, the Race to the Top Fund considers the extent to which a State “has ambitious yet achievable annual targets for increasing graduation rates.” Although this is a laudable goal, the guidance does not provide complementary instructions that ask the State to set similar targets for decreasing dropout rates. With the focus on four-year graduation rates, there are many incentives not to keep or re-admit students who would take longer than four years to graduate, who struggle academically, who are credit deficient, and who have left school due to pregnancy, homelessness, incarceration, illness, or other reasons. Without this instruction, the incentive to push poor performing students out of the system is magnified.

In accordance with this observation, the Forum recommends the following:

• Include language that requires states to set “ambitious yet achievable annual targets for decreasing school dropout rates overall and by student subgroup.”

In sum, the Forum believes that the funds provided by the American Recovery and Reinvestment Act represent an unprecedented opportunity to support what works for children and schools. Because this is a rare opportunity, it is essential that we provide appropriate incentives and think through likely consequences based on what the evidence shows to be true. For this reason, the Forum urges the Department to give serious consideration to the proposed changes included in this letter, and to the recommendations arriving from other sources that point out similar and related issues with the proposed guidance.

We look forward to partnering with the Department to execute new regulations that truly benefit our nation’s most disadvantaged children and schools. Thank you for your time and consideration.

Sincerely,

George Wood, Executive Director, The Forum for Education and Democracy

Sam Chaltain, National Director, The Forum for Education and Democracy

---

Endnotes

¹The Forum for Education & Democracy, Democracy at Risk: The Need for a New Federal Policy in Education (April 2008): iv, 2, 5-7, 12, 14-15, 38-40. For a summary, see L. Darling-Hammond and E. Rustique-Forrester (2005). The Consequences of Student Testing for Teaching and Teacher Quality. In Joan Herman and Edward Haertel (eds.) The Uses and Misuses of Data in Accountability Testing, pp. 289-319. Malden, MA: Blackwell Publishing. See also, Diamond, J & Spillane, J. (2002). High stakes accountability in urban elementary schools: Challenging or reproducing inequality? Evanston, IL: Northwestern University Institute for Policy Research; Firestone, W. & Mayrowetz, D. (2000). Rethinking "high stakes": Lessons from the United States and England and Wales. Teachers College Record, 102(4): 724-49; Herman, J. L. & Golan, S. (1993). Effects of standardized testing on teaching and schools. Educational Measurement: Issues and Practice, 12(4): 20-25, 41-42; Heubert, J. and Hauser, R. (eds.) (1999). High stakes: Testing for tracking, promotion, and graduation. A report of the National Research Council. Washington, D.C.: National Academy Press; Jones, M. G., Jones, B. D., Hardin, B., Chapman, L., & Yarbrough, T. M. (1999). The impact of high-stakes testing on teachers and students in North Carolina. Phi Delta Kappan, 81(3): 199-203; Lilliard, D. & DeCicca, P. (2001). Higher standards, more dropouts? Evidence within and across time. Economics of Education Review, 20(5): 459-73; Madaus, G., West, M.M., Harmond, M.C., Lomax, R.G. & Vator, K.A. (1992). The influence of testing on teaching math and science in grades 4-12. Chestnut Hill, MA: Center of Study of Testing, Evaluation, and Educational Policy, Boston College; Pedulla, J.J., Abrams, L.M., Madaus, G.F., Russell, M.K., Ramos, M.A., & Miao, J. (2003) Effects of state-mandated testing programs on teaching and learning: Findings from a national survey of teachers. Boston: National Board on Testing and Public Policy, Boston College; Whitford, B.L. & Jones, K. (2000). Kentucky lesson: How high stakes accountability undermines a performance-based curriculum vision. In B.L. Whitford & K. Jones (Eds.) Accountability, assessment, and teacher commitment: Lessons from Kentucky’s reform efforts. Albany, NY: State University of New York Press.

²Daniel F. McCaffrey, Daniel Koretz, J. R. Lockwood, Laura S. Hamilton (2005). Evaluating Value-Added Models for Teacher Accountability. Santa Monica: RAND Corporation.

³Henry Braun, Using Student Progress to Evaluate Teachers: A Primer on Value-Added Models (Princeton, NJ: ETS, 2005), p. 17.

⁴Darling-Hammond, L., & McCloskey, L. (2008). Assessment for Learning Around the World: What Would it Mean to be Internationally Competitive?” Phi Delta Kappan, Vol. 90, No. 4:263-272.

⁵Heubert, J. and Hauser, R. (eds.) (1999). High stakes: Testing for tracking, promotion, and graduation. A report of the National Research Council. Washington, D.C.: National Academy Press; Jacob, B.A. (2001). Getting tough? The impact of high school graduation exams. Education and Evaluation and Policy Analysis 23 (2): 99-122; Lilliard, D. & DeCicca, P. (2001). Higher standards, more dropouts? Evidence within and across time. Economics of Education Review, 20(5): 459-73;

⁶Democracy at Risk, iii, vi, 4-5, 15,17, 21.

⁷See, for example, Easton-Brooks, D. & Davis, A. (2009). Teacher qualification and the achievement gap in early primary grades. Education Policy Analysis Archives, 17 (15). http://epaa.asu.edu/epaa/v17n15/; Clotfelter, C., Ladd, H.F., & Vigdor, J.L. (2007). Teacher credentials and student achievement in high school: A cross-subject analysis with student fixed effects. Cambridge: National Bureau of Economic Research. http://www.nber.org/papers/w13617; Darling-Hammond, L., Holtzman, D., Gatlin, S.J., & Heilig, J.V. (2005). Does teacher preparation matter? Evidence about teacher certification, Teach for America, and teacher effectiveness. Education Policy Analysis Archives, 13 (42). http://epaa.asu.edu/epa//v13n42/; Boyd, D., Grossman, P., Lankford, H., Loeb, S., & Wyckoff, J. (2006). How changes in entry requirements alter the teacher workforce and affect student achievement. Education Finance and Policy, 1 (2): 176-216; Laczko-Kerr, I., & Berliner, D. (2002). The effectiveness of Teach for America and other under-certified teachers on student academic achievement: A case of harmful public policy. Education Policy Analysis Archives, 10 (37). http://epaa.asu.edu/epaa/v10n37; Darling-Hammond, L. (2009). Educational Opportunity and Alternative Certification: New Evidence and New Questions. Stanford, CA: Stanford Center for Opportunity Policy in Education. http://edpolicy.stanford.edu/pages/pubs/pub_docs/mathematica_policy_brief.pdf

⁸Boyd, D., Grossman, P., Lankford, H., Loeb, S., & Wyckoff, J. (September 2008). Teacher preparation and student achievement. NBER Working Paper No. W14314. National Bureau of Economic Research. Available at SSRN: http://ssrn.com/abstract=1264576. See also Darling-Hammond, L. (2006). Powerful teacher education: Lessons from exemplary programs. San Francisco: Jossey-Bass.

⁹Ibid 7.

• Education Policy

• George Wood's blog