Can Statistics Lie? How the Risk Matrix 2000 Influences Criminal Justice Outcomes in the UK

Since the mid-nineties, the judiciary system in the United Kingdom has utilised a tool called the ‘Risk Matrix 2000’ (RM2000), to determine whether an individual convicted of a crime will be likely to reoffend if given the opportunity. The outcomes of this tool are used to calculate the likelihood of recidivism in individual cases, and therefore influences how long certain convicts will be incarcerated for, for example, or whether they may have the possibility to be released on parole. David J. Cooke, who is a professor of Forensic and Clinical Psychology at Glasgow Caledonian University and who has recently been named Adjunct Professor at La Trobe Law School, spoke about his 2013 article ‘The Generalizability of the Risk Matrix 2000: On Model Shrinkage and the Misinterpretation of the Area under the Curve’ at La Trobe University in November last year. In this presentation, Professor Cooke spoke about the effectiveness of the use of the RM2000, while using data from a 2010 study[1]. In his view, conclusions derived from the use of the RM2000 are often misleading. For example, in his article, he mentions that “the prediction based on the RM2000s that a Very High Risk offender will reoffend will be wrong 93% of the time.”

So, why are these margins so big? Part of the problem in Professor Cooke’s eyes is the scoring system used by the Risk Matrix 2000. The model relies on a number of risk factors to determine whether an individual is considered to fall in a low, medium, high or very high category for reoffending. The process to score a convict on appears remarkably simple:

The RM 2000 comes in two forms: the RM2000s, is concerned with sexual reconvictions, the second form, the RM2000v, is concerned with violent reconvictions. The RM2000s is scored in two steps. For the first step, three risk factors are considered: Age of offender on release, number of sentencing occasions for a sexual offense, and number of sentencing occasions for any criminal offense. Scores on these three risk factors are used to assign the individual to one of four preliminary risk categories: Low, Medium, High, or very high. For the second step, four aggravating factors are considered: 1) Is there any male victim of a sexual crime? 2) Was any victim a stranger to the offender? 3) has the offender never been in a stable relationship for two years or more? And 4) have there been any non-contact sex offences?

In Professor Cooke’s view, part of the problem creating incorrect outcomes are the risk factors used in the scoring process. While testing the acquired data regarding “males convicted of a sexual offence and who had been ‘at risk least two years’, meaning there was two years’ worth of follow-up data”, he discovered that several risk factors used – like a convict’s age or their marital status -weren’t particularly statistically significant. Furthermore, he found that assessing individuals by these variables the relative risk of reoffending calculated in the group of very high risk offenders was huge – a whopping 274%, – while the absolute risk was only 5.2%. What this means, as Professor Cooke illustrates, is that when looking into a Very High Risk group of one hundred prisoners for example, only seven of them would go on to reoffend. But when one is operating by the principle of relative risk, as does the RM2000, this means that in order to prevent one reconviction, an astounding 14 prisoners would have to be locked up and removed from the community.

Looking into more similar statistic examples, Professor Cooke rightly poses the question whether evidence based on the RM2000 should be acceptable in judicial and quasi-judicial settings in the United Kingdom, when the uncertainty in predicting events is so large to begin with. The risk in using this model is that decision makers can be misled, and do not always understand the statistical nuances in relation to groups of people and individuals. Professor Cooke argues that this information should aid decision makers in making an informed decision rather than mislead them, and should be more probative than prejudicial. As Professor Cooke summarises, if statistical findings in certain groups are incorrect for more than ninety percent of the time, perhaps these findings shouldn’t be given any weight at all?

On reflection, should we be surprised by these findings? Consider the complex behaviour we are trying to predict – sexual offending. […] Even if we consider only one legal category – rape – it is clear that we are attempting to predict a behaviour underpinned by a complex array of factors – known and unknown – both inherent and external to the perpetrator. […] Is it really surprising that a few nonspecific variables fail to provide meaningful information about reoffending?

But what if the criminal justice system was to step away from the RM2000 models? Is there a worthy alternative? According to Professor Cooke, a significant step in the right direction would be the employment of guidelines based on best empirical evidence and clinical evidence, to assist with the decision-making process, rather than trying to predict the likelihood of an individual’s relapse. While several researchers have already been pushing for shifting the focus in violence risk assessment away from making probalistic statements, there is still a long way to go. But in Professor Cooke’s view, these changes can’t be pushed through soon enough: “Sexual violence is a complex human problem, ARAIs [like the RM 2000] may, to some, appear to be both neat and plausible, but unfortunately they are wrong – and dangerously so.”

To read more about Professor David J. Cooke’s research, please refer to his publications.

[1] Barnett, G. D., Wakeling, H. C., & Howard, P. D. (2010).

An examination of the predictive validity of the Risk Matrix 2000 in England and Wales. Sexual Abuse: A Journal of Research and Treatment, 22, 443–470. doi:10.1177/1079063210384274

La Trobe