Swift Certain Fair Resource Center

  • Programs
  • SCF Evaluations
  • Newsroom
  • About
Home
|
Newsroom
|
How Powerful Is the Evidence in Criminology? On Whether We Should Fear a Coming Crisis of Confidence
Posted on July 23, 2018 by Kelly Smith

How Powerful Is the Evidence in Criminology? On Whether We Should Fear a Coming Crisis of Confidence

Arguably the most risky kind of evidence is a large effect size gleaned from a statistically underpowered study. When we see findings like this, our reaction should probably be one of “organized skepticism” (Merton, 1973)—where caution is exercised and a call for replication is made before we get too excited about the single study (see also Kulig, Pratt, and Cullen, 2017; Pratt, Turanovic, and Cullen, 2016). But demonstrating such caution is not something humans are either wired for or socialized to do very well (Kahneman, 2011). And as a discipline, criminology has a sketchy track record with demonstrating caution in the wake of a splashy new finding.
Take, for example, the recent treatment—both in policy and empirical circles—of the correctional intervention known as Project HOPE. Nearly a decade ago, a single evaluation (one with a relatively low level of statistical power) of a correctional program in Hawaii found that threatening offenders on probation with a short but immediate stint in jail for even minor technical violations substantially reduced recidivism (Hawken and Kleiman, 2009). On the coattails of this single small study, HOPE-like programs spread rapidly to over a hundred locations in dozens of states, and has even gone international (Bartels, 2017). But then replications of programs based on the HOPE model started to trickle in—studies with the kind of statistical power the original evaluation lacked (see, e.g., Hamilton et al., 2016)—and the results were unequivocal: HOPE programs had no appreciable effect on recidivism (Lattimore et al., 2016; O’Connell, Brent, and Visher, 2016). That thousands of offenders, victims, and community members have been subjected to the consequences of what now appears to be a failed program underscores the importance of exercising caution when low-powered studies reveal startling results (Cullen, Pratt, and Turanovic, 2016). This is the very thing that an evidence-based policymaking philosophy should seek to avoid.

Posted in Newsroom
Crime rates and Alaska criminal justice reform
HOPE Probation: A New Path to Desistance?

SIGN UP FOR UPDATES

Keep up with the latest from the Swift Certain Fair Resource Center. Subscribe to our newsletter.

Find us

Swift Certain Fair Resource Center New York University 370 Jay Street, 12th Floor Brooklyn, NY 11201 Telephone: (646) 308-0508

E-mail: info@scfcenter.org

New York University Marron Institute of Urban Management
Web Site Notice of Federal Funding and Disclaimer:
This Web site is funded in whole or in part through a grant from the Bureau of Justice Assistance, Office of Justice Programs, U.S. Department of Justice. Neither the U.S. Department of Justice nor any of its components operate, control, are responsible for, or necessarily endorse, this Web site (including, without limitation, its content, technical infrastructure, and policies, and any services or tools provided). SCFCenter.org - Copyright © 2023. All rights reserved.