“There are three kinds of lies: lies, damned lies, and statistics.”

There are many potential pitfalls when working with data and statistics, but one common problem occurs when the data is not representative of the whole thing being examined and “cherry picking” is one way that can happen. This is sometimes also called “counting the hits and ignoring the misses.”

When researching a topic, it can be tempting to focus on the data and evidence that supports your thesis, so care must be taken to look at and weigh all of the available evidence and not just the parts that you like. Psychics often use this technique to promote themselves. They will make a string of (often very vague) predictions, one or two of them will happen to be correct, and then they will use those successes as evidence of their ability, ignoring the fact that they were wrong more often than not.

The easiest way to prevent this is to try to disprove your own thesis. Specifically look for data that contradicts your other evidence. If you find any, you need to address it in your writing. You may need to concede a point, you may need to qualify your thesis statement, or you might include an analysis of the contradictory data along with an explanation of why it isn’t sufficient to negate your other evidence. And sometimes, you might just have to admit that you were wrong.