Criminal Justice Reform Guided by Evidence: Social Control Works
link.springer.com/article/10.1007/s11292-023-09558-w
The field of experimental criminology has become influential in designing, testing, and evaluating criminal justice reforms and reporting their results, even when results are contrary to popularly held beliefs. After all, science advances by trial and error and generating findings sometimes that are at odds with prevailing beliefs. Science is about generating objective. The only faith required is the belief in the scientific method. Findings should be objective as possible, recognizing that choices of statistical analyses and program evaluation designs can change findings. Hence the need for replication of results in other settings and reproductions of results when new methods develop.
A good example is the demonstration trial of the Honest Opportunity Probation with Enforcement (HOPE) program in Hawaii. Despite much heralded efficacy of the HOPE program in Oahu, Hawaii in reducing recidivism, a four-site demonstration experiment in other states failed to find evidence of efficacy in reducing criminal offending (Lattimore et al., 2016). Clearly, the HOPE program was politically popular at the time because it offered a way to reduce prison for repeat offenders through a more focused use of short-term jail for probation violations. The Hawaii program also fits Kleinman’s [sic] suggestion that punishments should be “swift and certain, rather than severe” (Kleiman, 2009, p. 3). Publication of the null findings from the HOPE replication was a setback for the enthusiasm of expanding this program around the country, and provides a good example of how science can inform policy. At least the HOPE program merely proved to be not as efficacious as initially estimated; other programs may suffer from the flaw of actually being criminogenic for participants or increasing crime rates. In the context of criminal justice reform, social programs that do not show evidence of being effective at reducing crime or that increase crime should be redesigned or abandoned, regardless of political popularity.
Challenges and Prospects for Evidence-Informed Policy in Criminology
annualreviews.org/doi/abs/10.1146/annurev-criminol-022422-124116
Indeed, experiments have high internal validity, yet external validity and scalability are fundamental to effective policy (Nagin & Sampson 2019, Sampson 2010). Even with evidence from a high-quality RCT (and with same-site replication), the same effects may not transfer to other areas, populations, or sociocultural contexts. A recent example of this is Project HOPE (Hawaii’s Opportunity Probation and Enforcement program)—a program based on “swift, certain, and fair” sanctioning for probation violators—which showed remarkable effects on reducing recidivism in Hawaii but nowhere else (Cullen et al. 2016, 2018; Lattimore et al. 2016). RCTs can do well to establish whether a treatment or program works somewhere, but policymakers and practitioners need to know whether “it will work for us” (Cartwright 2011, p. 767)—a judgment call that always involves some amount of guesswork.
Court Delays and Criminal Recidivism: Results from Danish Administrative Data and a Policy Reform
tandfonline.com/doi/full/10.1080/07418825.2023.2260451
Swiftness of punishment also plays a significant role in probation practices promoted by project HOPE (Honest Opportunity Probation with Enforcement). HOPE focuses on providing swift, certain, and fair responses to violations of probation conditions for drug offenders in the US. Following promising initial results that echo the expectations from Deterrence Theory, the project was soon expanded to other areas than Hawaii where it started and to other (non-drugs) types of offenders (see Cullen et al., for a discussion of the expansion of HOPE). Based on the initial results from evaluations of HOPE (e.g., Hawken & Kleiman) we should thus expect time to adjudication to matter for the risk of new crimes, in the sense that longer time to adjudication increases the crime response. A recent multisite evaluation of HOPE later found, however, the discouraging result that HOPE was unlikely to have impacted rates of criminal recidivism to any meaningful degree (Lattimore et al.).
Cause, Effect, and the Structure of the Social World
papers.ssrn.com/sol3/papers.cfm?abstract_id=4445710
This Essay is built around a central empirical claim: that most reforms and interventions in the criminal legal space are shown to have little lasting effect when evaluated with gold standard methods. While this might be disappointing from the perspective of someone hoping to learn what levers to pull to achieve change, I argue that this teaches us something valuable about the structure of the social world. When it comes to the type of limited-scope interventions that lend themselves to high-quality evaluation, social change is hard to engineer. Stabilizing forces push people back towards the path they would have been on absent the intervention. Cascades—small interventions that lead to large and lasting changes—are rare. And causal processes are complex and context-dependent, meaning that a success achieved in one setting may not port well to another.
This has a variety of implications. It suggests that a dominant perspective on social change—one that forms a pervasive background for academic research and policymaking—is at least partially a myth. Understanding this shifts how we should think about social change and raises important questions about the process of knowledge generation
**
Multi-site replication attempts using RCTs continue to be rare. One recent example pertains to “swift, certain, and fair” sanctioning as developed in Project HOPE. “Swift, certain, and fair” refers to a model in which a probation violation (e.g., a positive drug test) results in a certain and immediate, but relatively mild sanction, such as 24 hours in jail. This replaces a more indeterminate system, in which an individual can accrue multiple violations before a judge decides to revoke probation. In such a system, probation revocation often leads to many months of incarceration. A randomized controlled trial showed that Project HOPE led to large reductions in both drug use and time-incarcerated, with long-lasting effects.
This study was exciting to many, not only because of its impressive effects, but also because it supported a set of theories that were gaining popularity at that time. A central theme in behavioral economics is that people are myopic, meaning that they are expected to respond more to the threat of short sentences that would go into effect immediately than to long sentences that might go into effect eventually. By altering criminal justice sanctions to better correspond with behavioral incentives, observers hoped to be able to reduce drug abuse without using big-stick carceral sentences. The Project HOPE evaluation was backed with a theory of human nature that made its results seem broadly generalizable. It took off in a rise that observers described as “meteoric”; within a few years, “swift, certain, and fair” sanctioning had been adopted in at least 160 jurisdictions. Again, the National Institute of Justice funded randomized controlled trials to try and replicate Project HOPE’s success across five sites. The results were not promising: “swift, certain, and fair” sanctioning did not offer any detectable improvements over the status quo.116 While jurisdictions may continue to operate in a HOPE-like fashion, the balloon of optimism has largely deflated.
Few replication attempts have been as exhaustive as the HOPE ones; in part, because few initial studies have been as compelling.
116 The original success and subsequent failure to replicate has sometimes been credited to the motivation and enthusiasm of the original judge behind Project HOPE. A more recent RCT was conducted in Oahu with this original judge. The authors found some tentative successes: the treatment group had fewer failed drug tests, and fewer arrests involving a new criminal charge. Janet Davidson et al., Managing Pretrial Misconduct: An Experimental Evaluation of HOPE Pretrial 4, 6 (Goldman Sch. Of Pub. Pol’y, Working Paper, Jan. 2019), gspp.berkeley.edu/assets/uploads/research/pdf/HOPE_final_evaluation_January_2019.pdf. However, the overall arrest rate and the number of jail days served was equivalent across treatment and control. Id. At 5- 6. The outstanding successes of the original study were not replicated even with the same place and same group of people. The study remains unpublished.
Annals of Research and Knowledge: A blueprint for an evidence-based justice system
ark.allrise.org/high-risk/low-need/community-corrections/hope-probation The ARK is a searchable database containing information on evidence-based and promising programs for persons involved in the criminal justice system. Because no intervention works for everyone, programs are cataloged according to participants’ risk and need profiles and stage of processing in the justice system. For example, users can obtain information on programs suited for high risk and high need individuals at the pretrial stage, or low risk and low need individuals at the pre-arrest stage.
Effectiveness Studies
Research on the effectiveness of HOPE probation is decidedly mixed. A randomized controlled evaluation of the original HOPE program in Hawaii found that HOPE participants were 55% less likely than individuals on probation as-usual to be arrested for a new crime over 12 months from entry into HOPE or probation, 72% less likely to test positive for illicit drugs, 61% less likely to miss probation appointments, and 53% less likely to have their probation revoked (Hawken & Kleiman, 2009). Subsequent analyses determined that positive effects on recidivism lasted for at least 10 years after entry, which was several years after most participants had been discharged from probation (Hawken et al., 2016). Quasi-experimental studies in other jurisdictions have reported similar findings. Statewide implementation of the HOPE model in Washington State was associated with significantly lower rates of probation violations and convictions for new offenses over a follow-up period of 12 months (Hamilton et al., 2016). Similarly, quasi-experimental evaluations of pilot HOPE projects in several jurisdictions, including North Carolina, Virginia, and Fort Bend County, Texas, reported significantly lower probation revocation rates or rearrest rates for HOPE participants after one year (Council for State Governments, 2017; Snell, 2007; Virginia Criminal Sentencing Commission, 2016).
In stark contrast to these promising outcomes, a four-site randomized controlled trial involving more than 1,500 probationers in Arkansas, Massachusetts, Oregon and Texas counties found no improvements from HOPE over probation as-usual on rearrest rates over a period of two years from entry, larger numbers of probation revocations for the HOPE participants, and higher programmatic costs for HOPE stemming from increased drug testing, service of warrants, and provision of jail sanctions (Lattimore et al., 2016). Another randomized controlled experiment in Delaware similarly found no benefits from HOPE over 18 months on any outcome measure of recidivism, substance use, or probation violations (O’Connell et al., 2016).
Commentators have offered various explanations for the inconsistent results emerging from rigorous experimental trials. Discrepant outcomes have been attributed to inconsistent fidelity in implementing the HOPE replications (Hawkin [sic], 2016; Schiraldi, 2016), as well as conceptual flaws in the behavioral theory of deterrence underlying the HOPE model (Cook, 2016; Cullen et al., 2016). Researchers have also failed to consider the appropriate target population for HOPE. Sanctioning regimens may be effective for low need (i.e., nonaddicted) individuals whose drug use is largely under voluntary control; however, persons who are compulsively addicted to drugs or alcohol may be less responsive to threats of punishment, and in greater need of substance use treatment and other services to succeed (Marlowe, 2011). Further research is needed to identify the appropriate target population for HOPE as well as best practices that produce consistently favorable results.
Target Population
The target population for HOPE probation has not been identified empirically. The program was developed for high risk individuals who are likely to engage in multiple technical violations and have their probation revoked. Probationers are selected for inclusion in HOPE based on prior probation violations or revocations, or high scores on a standardized risk assessment instrument.
Because treatment is not a core component of HOPE, and the intervention relies predominantly on the threat of negative sanctions to reduce drug use, better outcomes are likely to be achieved for low need persons whose substance use is under voluntary behavioral control (Marlowe, 2011). Individuals who are compulsively addicted to drugs or alcohol are less likely to respond to threats of punishment, and will typically require intensive treatment and other social services to desist from substance use and drug-related violations. In Hawaii, high need probationers who are unable to remain abstinent in the HOPE program are transferred to drug court where they receive a considerably greater emphasis on treatment, positive incentives, and support from a multidisciplinary treatment team (Alm, 2013, 2015). A qualitative evaluation of the Hawaii HOPE program confirmed that many participants require intensive treatment, social services, and other supports to succeed, and a sanctions-based model is generally insufficient for these high need individuals (Bartels, 2016). Further research is needed to confirm this hypothesis and identify the appropriate target population for HOPE.
