Thursday, February 12, 2015

Indianapolis Council Committee Ignores Studies Showing that Pre-K Programs Offer No Lasting Benefits to Spend Millions on the Program

The Indianapolis Business Journal reports:
A City-County Council committee on Wednesday night backed a bipartisan plan to tap reserves and other funds to pay the city’s share of Indianapolis Mayor Greg Ballard’s plan to pay for up to 1,000 poor children to attend preschool.

The Community Affairs and Education Committee unanimously approved the plan. Using primarily set-aside money rather than the city’s general fund, the plan must also win support of the full Indianapolis City-County Council, which in December passed the framework for the preschool program by a wide margin.


The committee approved setting aside $4.2 million in city dollars to fund the first year of what is expected to be a five-year program. The city plans to use $2 million by tapping into the city’s fiscal stability fund, with $500,000 of that coming from interest, $1.7 million that was saved through a change to the homestead tax credit program and $500,000 from county option income tax, according to the city’s Office of Education Innovation. Both the Republican mayor’s office and several council Democrats have backed the plan.

The city’s investment is expected to be $20 million over five years. Another $20 million is expected to be raised by corporate and philanthropic donors.
In voting to spend millions of dollars on pre-K, city-county councilors ignored studies showing early childhood education doesn't work.  A independent and comprehensive study of Head Start commissioned by the Department of Health and Human Services found that the program offers no long term benefits.  Last year that 2005 study was updated to look at high v. low quality Head Start program and still found it made no difference...there was still no lasting impact.
Applying these analytic innovations to the experimental HSIS evaluation data, we find little evidence that Head Start’s impact varies systematically by the level of quality in the program for the available, limited quality measures. The frequency of statistically significant differences in impacts by quality levels is no greater than one would expect to observe by chance alone when no true differences exist. The one exception to this pattern is the discovery that, for 3-year-olds, lower exposure to academic activities is associated with more favorable short-run impacts on social development. There is almost no indication that either high or low quality Head Start in any dimension leads to Head Start impacts that last into third grade for either age cohort, consistent with the overall findings of the Head Start Impact Study not disaggregated by quality level.
A 2013 study by Vanderbilt University of Tennessee's voluntary pre-K program found the same thing:
The relatively large effects of TN‐VPK on the Woodcock Johnson achievement measures found at the end of the pre‐k year were greatly diminished and no longer statistically significant at the end of the kindergarten year. The only exception was a marginally significant negative effect on Passage Comprehension such that nonparticipants had higher scores at the end of the kindergarten year than TN‐VPK participants.

Similarly, at the end of first grade, there were no statistically significant differences between TN‐VPK participants and nonparticipants on the Woodcock Johnson achievement measures with one exception. There was a significant difference that favored the nonparticipant group on the Quantitative Concept subscale.

These diminished effects were not entirely unexpected in light of the findings in other longitudinal studies of the effects of early childhood programs on economically disadvantaged children. For preschool programs, a typical finding is that the cognitive effects are not sustained for very long after that initial year. Though none of those other studies investigated the effects of a single year of a scaled up state‐funded public pre‐k program, many involved even more intensive programs that nonetheless failed to show effects on cognitive achievement measures that were sustained for very long. Like TN‐VPK, however, these programs did not involve any continuous, focused support in subsequent years for sustaining the gains made during the initial program year.
Grover J. Whitehurst of the liberal Brookings Institute concerned about the "weak evidence behind the groundswell of advocacy for public investments in statewide universal pre-k" wrote an article in 2014 discussing the pre-K advocates' misuse of unscientific studies purporting to show the programs work:
The previous tables and descriptions refer to 13 separate studies (including 3 similar studies of district programs and two similar studies of statewide programs in Oklahoma and Georgia).  Of these 13, six report enduring and meaningful impacts beyond the pre-k year, four report null, negative, or very small positive impacts beyond the pre-k year, and three do not report findings beyond the pre-k year.

It would be easy for someone without the training to carefully evaluate these studies or someone with a strong motive to advocate for the expansion of publicly funded pre-k to summarize this research by saying that the preponderance of evidence supports universal pre-k for four-year-olds. After all, of the 10 studies I’ve reviewed that have long-term follow-up, 60 percent report substantive positive outcomes.[xiii]  Libby Doggett, the Obama administration’s point person on Preschool for All, has been singing exactly this song at every opportunity:
You have to look at the preponderance of the evidence. Better high school graduation rates, social and emotional stability, less crime and other results speak for themselves.[xiv]
But results do not speak for themselves.  Rather, it is the combination of results and the research designs that produce them that do the speaking.  And some of the combinations speak a lot louder than others.

Not one of the studies that has suggested long-term positive impacts of center-based early childhood programs has been based on a well-implemented and appropriately analyzed randomized trial, and nearly all have serious limitations in external validity.  In contrast, the only two studies in the list with both high internal and external validity (Head Start Impact and Tennessee) find null or negative impacts, and all of the studies that point to very small, null, or negative effects have high external validity.  (Emphasis supplied.)  In general, a finding of meaningful long-term outcomes of an early childhood intervention is more likely when the program is old, or small, or a multi-year intervention, and evaluated with something other than a well-implemented RCT.  In contrast, as the program being evaluated becomes closer to universal pre-k for four-year-olds and the evaluation design is an RCT, the outcomes beyond the pre-k year diminish to nothing.

I conclude that the best available evidence raises serious doubts that a large public investment in the expansion of pre-k for four-year-olds will have the long-term effects that advocates tout. 


Anonymous said...

why oh why doesn't Indy ever confer with you before doing anything

Paul K. Ogden said...

Anon 1:41, actually the question is why the council doesn't care about taxpayers enough to listen to credible studies that demonstrate a program they're going to spend millions on doesn't work.

Anonymous said...

Yes, Paul... that IS the question. And we more and more over-taxed impoverished Marion County voters want it answered especially by Zach Adamson, Marilyn Pfisterer, Mary Moriarty Adams, Ben Hunter, Mike McQuillen, Maggie Lewis, Aaron Freeman, Jeff Miller, Joe Simpson, and Jack Sandlin (to name a few)...

Anonymous said...

Paul, using psycho-educational testing jargon, these temporary gains for Pre-K program students including Headstart are termed the "hot house affect" whereby a flower may indeed bloom quickly in a hot house; however, the forced bloom does not last as long as the bloom for a flower left to its own schedule in a naturally occurring healthy environment.

Anonymous said...

Paul, as an addendum to my comment from 3:48 PM, I remain stymied by the failure of our elected officials to acknowledge the significance and relevance of the empirical research investigations, peer reviewed as per publication in journals and publications. These are the same officials who tout research-based methods, but yet fail to consider the existing research based completely on the validity and reliability of the measuring instruments that are used.

The Tennessee investigation used the Woodcock Johnson, an individually administered assessment that requires licensing qualification for the administrator. The Woodcock Johnson is not a group test where some teacher assistant throws down a paper test on the desk top of each student in a classroom. It is a clinically individually administered assessment instrument that yields valid results, perhaps results that our local officials do not wish to accept based upon their warm fuzzy feelings about pre-K programs.

Anonymous said...

What about these studies and evaluation from a bipartisan group?

Paul K. Ogden said...

Anon 1230, as the liberal Brookings Institute person noted those studies all have methodological flaws. Not every study is the same in terms of validity. It's not hard to commission a study that will give you the result you want.

Anonymous said...

Anonymous @ 12:30 PM, I spent considerable time reviewing your link; however, I'm left with the conclusion that evidence-based, your term, is not the equivalent of empirical research conducted by a principal investigator operating in a clinical setting.

The link you provided only validated my statement that the Head Start program has not yielded the results expected during its design phase. Locally, as in Central Indiana, Head Start programs are staffed by unqualified individuals, meaning unqualified and untrained and certainly not licensed/certified professional preschool educators. It's a form of free childcare where the employees generally are nice folks, but are clueless about early childhood education.

As for the other evidence-based information provided in your link, all I managed to derive from reading it was that a few programs have a carryover into adult life where criminal activity may, or may not, be curtailed as a result. Now, if we want to curtail later criminal behavior or being on the wrong side of the law, then I'd recommend social programs. Public school education simply is not designed or intended to cure social ills; that's within the purview of the social workers. Do not burden the public schools with yet another mandate or directive.