Take crime statistics with a grain of salt. They are usually generated by people with a vested interest in their values, who are probably unqualified to create them, and almost no one can verify them independently. When they are reported, it is typically without a description of their methodology. Although they have their uses, they are almost always in narrow contexts.
Almost all crime statistics come from police and from courts: they include reported crimes, convictions, and similar by-products of official activities. They don't come with warning labels, such as “these numbers may be harmful to society”, as they probably should. When proper scientific papers include data, those papers explain the methodology by which the data were collected. But not crime statistics, which are often more political than scientific.
Those gathering and reporting them don't even ordinarily include estimates of the accuracy of the statistics. Even incompetent and biased pollsters usually say things like, “These figures are accurate to within three percent”, or something like that. You may not know what that means, even the pollsters may not know what that means, but at least they make an attempt.
Consider the possibility of random error. Witnesses are notoriously unreliable (more on that below), and anything derived from witness reports ought to be qualified. To illustrate, let's suppose that we have a population that's ten percent black and ninety percent white, and they're all unprejudiced and honest folk — except for the criminals — who sometimes make mistakes. (I'm inventing these numbers because the point isn't about the numbers themselves, and I'd like round, simple numbers that are easy to work with.) Suppose that there are a hundred crimes, that blacks and whites are equally likely to commit crimes, that each crime has a witness, and the witnesses make errors one time out of ten. We ask them the colour of the perpetrator.
For the one hundred crimes, we'd expect ten by blacks and ninety by whites, right? Wrong. Simply because of honest, random error, ten percent of the white perpetrators will be reported as black, and ten percent of the black perpetrators will be reported as white. The result will be 82 of the witnesses will report white perpetrators, and 18 will report black. The black crime rate will be reported as almost twice what it really is.
How often have you heard estimates of witness error? Such errors are very common. A standard feature of criminal justice classes is for the teacher or professor to stage a crime in front of a class, without warning the students in advance of what's being done. For instance, during a lecture, an actor will enter the room and steal something from a desk unobtrusively, or maybe the actor will be obvious about it, and possibly a second actor will do something else to distract the students. Then the students will be asked about what they have witnessed. It's not uncommon for students in such sitations to be mostly wrong about what the perpetrator was wearing, what was taken, and so on. Of course, those that go on to work in the criminal “justice” system often forget the lesson.
Witnesses can be biased by the questioner. If you offer hints to witnesses, they are likely to take them. For instance, if you ask a witness if a perpetrator was white, you'll get more “white” answers than if you ask if he or she was black, and conversely. Pollsters know this, and the honest ones will, with multiple choice questions, randomize the order of the possibilities to eliminate that error. How many cops randomize their questions to potential witnesses?
Even unintentionally, questioners can convey bias to witnesses. Scientific research uses double-blind techniques to prevent this. For example, in a controlled drug study, patients might be given, randomly, one of various drugs or placebos to test their effects under certain conditions. Researchers then examine the patients to see if which patients have improved or become worse. Neither the patients nor the examiners know who received which drugs: that's why it's called a “double”-blind study; it is well known that the expectations of the examiners can bias the outcome of the study. It's even common to use double-blind techniques when testing animals or inanimate objects. Scientists know not to trust studies which don't use double-blind procedures. How often are police questioners kept in the dark about a case before being sent in to examine witnesses?
It's also easy to implant memories into subjects. For instance, if a witness is asked, “Was the thief driving a white pickup truck?”, the witness may come to believe that the thief was driving a white pickup truck, even if the thief was, in fact, driving a blue sedan. The techniques for planting memories can even be taught, though they usually don't work; however, under certain circumstances they have a high success rate. Two of the main factors are the relationship between the questioner (the cop) and the level of trust that the witness has in the questioner.
Studies have show that these memories can be deep and completely indistinguishable from memories of real events. For instance, if mothers tell their children about something which happened when the child was young, the child may come to believe it. Afterwards, as adults, the children with the implanted memories can't be distinguished from children with real memories, even using so-called truth drugs, hypnosis, and psychological techniques. Fake memories can be as real as real memories.
Police consciously try to gain the trust of witnesses, if only to get more information from the witnesses. However, a trusting relationship is precisely the best environment for implanting memories. If the police ask leading questions (“Was he driving a white pickup?”) then memories can be implanted, and they might never be separated from real memories in a courtroom or even by a psychologist in a clinical setting. If body camera footage of police questioning witnesses were routinely released, it would show a great amount of sloppy police work, including but not limited to demonstration of police potentially and actually biasing witness testimony.
Crime reports are almost always written up by police. There aren't independent crime investigators in the statistics. One consequence of this is selective reporting: writing up reports more or less frequently depending on the class of persons reporting the crimes and the class of persons observed to be possible perpetrators. Poor neighbourhoods may not get so much attention as wealthier ones, and cops might not find some victims so credible as others. The bias in reporting can also lead to a counter-intuitive result: adding police to a neighbourhood can sometimes increase the reported crime rate, because there are more police to generate reports.
Oversimplifications can also be extremely misleading. The crime rate is higher in cities, where the population density is higher. The crime rate is higher when education levels are lower. The crime rate is higher when incomes are lower. So when you see a statement such as “Blacks have higher crime rates than whites” is it because blacks live in more densely populated areas than whites, because blacks have lower education levels than whites, because blacks have lower incomes than whites, or some other factor? Or have these values been factored out? Is the sample adjusted for age and gender? (Young men have the highest crime rates, but these factors — age and gender — can have different distributions between blacks and whites in various sample groups.) Is the sample adjusted for employment level? Is the sample adjusted for immigration status? These factors both affect crime rates. In other words, such a simple statement, that one race has crime rate X while another has crime rate Y is meaningless without qualification.
When someone uses an oversimplification such as the above, he or she is likely to make incorrect inferences. It's easy, but incorrect, to say that, “Because blacks are X times more likely to commit crimes, it's X times more likely that a black person did this thing” Not one in ten thousand people, including cops and police, could tell you the formula for the correct calculation when multiple factors are involved. In fact, even if the formula were in front of them, probably not one in ten thousand would even recognize the formula or know how to plug in the numbers to estimate the correct probability given the factors. Cops self-select for violence and authoritarian behaviour, not for statistical acumen. No, the typical person carries an invalid oversimplification in his head and uses the wrong number in an erroneous manner, multiplying the error.
These errors don't simply add up, they can multiply. For instance, if a person has a ten percent disadvantage as a suspect, a ten percent disadvantage as a litigant, and a ten percent disadvantage during sentencing, what is his overall disadvantage? It isn't 30 percent (10 plus 10 plus ten), it's 33 percent (1.1 times 1.1 times 1.1, minus 1).
Surveys of existing data show that blacks are more likely than whites to be convicted when brought to trial. Those same data feed the crime statistics, which are largely based on convictions. Then those same numbers are used to justify harsher treatment of blacks (“they're more likely to commit crimes”), which is to some extent a circular argument. Yet, these are the numbers you will read in the papers and hear on television.
So far, I haven't considered dishonesty or prejudice. All of the above sources of error are based on wrong use of mathematics, on logic errors, on inadequate knowledge of science, or simply on bad police procedure. If we throw racism, bias, and prejudice into the mix, then we greatly compound the errors.
The system has several feedback loops, which tend to amplify the disadvantages of minorities such as blacks or hispanics or muslims, even more so when prejudice is involved. If someone has a disadvantage because of his race or other attribute, then he is more likely to be a suspect because of prejudice and simple error, he is more likely to get poor representation (and even more likely if he is poor), more likely to have judgment rendered against him, and more likely to get a harsher sentence. This makes the statistics worse, which increases the disadvantage of the next victim of the system. Each time a black (or hispanic or other) person is treated worse by the system, it increases the disadvantage for the next person in the same group: the cops, the witnesses, the prosecutors, and the judges all have the perception, subliminal or not, that members of the group are more likely to be guilty. Overall, given that a black person is more likely to be poor, but that people see the guilt as more attributable to race than to poverty, the system error is exaggerated. And so on...
(To make matters even worse, the system itself creates some of the crime. For instance, harsh sentences actually increase the likelihood of recidivism, so that, by sending people to so-called correctional institutions that aren't correctional at all, the courts — even if it's merely as a consequence of the laws they must work with — may increase the crime rate.)
There are no proper, scientific crime data. The data all come from a broken criminal justice system: there are no control groups, there is no way to reproduce the results, and there are no adequate records to audit the process which generated the numbers in the first place. The only way to be confident that the numbers are correct would be to requestion a sample of all the witnesses, to re-try a sample of all of the cases, and so on, all under carefully controlled conditions, and that has never happened. That will never happen. So crime statistics derived from the system itself can't prove a damned thing about the system, whether it works correctly or whether the numbers are right or wrong.
You know that tape which police use to fence of crime scenes to prevent damaging the evidence before it can be examined? The biggest destroyers of evidence, including pollution of witness and public perceptions, may be the police, prosecutors, and judges themselves. How is it possible to verify that a witness is testifying correctly after a cop has used sloppy techniques to question that witness? It can't be done.
Finally, but not least, most convictions in the United States do not result from a trial: they come by way of plea bargains. In a plea bargain, a prosecutor makes a deal with a suspect: “If you agree to plead guilty to a lesser charge, then we won't prosecute for a greater charge”. This practice, is, on its face, unethical and dishonest: among other things, the prosecutor is either lying or he's not upholding the law. If a suspect is guilty of a greater crime, then what is the justification for not prosecuting him as the law demands? If he or she has the evidence, he ought to prosecute, else he is betraying the trust of the people, and if he doesn't have the evidence, then he is dishonest by threatening without cause. Worse, the threat which creates plea bargains is used mostly against poor defendants, who are disproportionately black or otherwise non-white. Poor defendants tend to have inadequate resources to defend themselves, guilty or not, and usually cannot even get good advice. They know the deck is stacked against them, and are more likely than better off suspects to respond to the threat and to accept the deal which, for them, isn't always a real bargain. Conversely, well-heeled and well-connected suspects can arrange for lighter charges in return for favours such as campaign contributions to prosecutors and preferential terms in private transactions. Either way, since most convictions arise from plea bargains, the quality of statistics derived from convictions is highly suspect.
Ignoring, for the moment, the unfairness of the system to individual persons, the overall result is that the crime statistics, even when corrected for factors such as income level, education, population density, and other variables, can be wildly off the mark.
So the next time you read that, say, blacks have a crime rate X times higher than whites, or one place has a crime rate Y times that of another place, or anything else like that, don't believe what you read. The numbers are almost certainly wrong. It's not a question of “if they're wrong”, it's a question of “how much they're wrong”.
Edit History:
Friday 2017.10.06 — Initial release.
Comments, criticism, and suggestions are welcome. Please direct them to buffalo-bull-editor@tatanka.com. If your remarks pertain to a specific issue or article, please identify the issue or article in your e-mail. We reserve the right to publish comments unless you explicitly request otherwise; they may be edited for spelling, grammar, or length. You will not be identified unless you say it is OK; your e-mail address will not be published unless you say that is OK, too.
The Buffalo Bull is a free e-mail newsletter published by Michael Marking. For subscription information, and for current and past issues, please visit http://www.tatanka.com/buffalo_bull/index.html
Copyright 2017 Michael Marking. All rights reserved, including moral rights.
This newsletter may be redistributed freely, in paper or electronic form, but only if distribution is done without changes, abridgement, or amendment. Specifically, this work is licensed under the Creative Commons Attribution-NoDerivs 3.0 Unported License. Proper attribution is made by leaving these end notices attached to the newsletter. To view a copy of this license, visit http://creativecommons.org/licenses/by-nd/3.0/ or send a letter to Creative Commons, 444 Castro Street, Suite 900, Mountain View, California, 94041, USA.