We know that inferential statistics helps people to make intelligent and accurate conclusions about a greater population based on analysis results of a small sample. Simply put, we can make estimations about populations based on a small sample of people.
One example: if we met a small group of doctors and find that the cardiologists among them earned more than general physicians, we could infer that cardiologists, generally, earned more than physicians.
Another example: In exit polls, the pollsters ask a small group of people at polling stations about who they have voted. Based on their responses, the pollsters make a generalised estimation on who is likely to win from that constituency.
However, there can be two problems in here.
- Isolated evidence
- Random variation
Isolated evidence
The problem of isolated evidence happens when we draw inferences based on only a few cases. Such inferences might not be accurate.
Like the above example, if we happen to know only the top cardiologists who earn high salaries, we might be tempted to generalise that all cardiologists earn high salaries. This is because we personally know a few that earn high salaries. Here, we have isolated evidence of only a few known cardiologists that do not represent the entire population of cardiologists.
In case of isolated evidence, we generalise based on known cases. So, there is an element of cognitive bias. Since cognitive biases strongly influence our decisions, we tend to generalise based on these cognitive biases. It influences so much that we look at every evidence in the light of our cognitive biases.
This problem is more likely to occur in the context of personal experiences.
For example, if we do not have a good experience of a certain product (or a service or an institution), we will desist our friend from using it. Ours may be a case of isolated evidence of bad experience with that product (or service or person). Most other people might have had other experiences. In short, our isolated evidence is not enough to conclude whether something is good or bad.
Random variation
The problem of random variation happens when we have insufficient sample data which may not be representative of the population. Here, we are likely to make inaccurate predictions about the entire population based on inadequate data. In such cases, any observed trend is out of randomness.
Random variation is independent of the effects of cognitive and systematic biases. We must aim to collate sufficient data points to nullify the effect of random variation. In general, the larger the sample size, the smaller the effect of random variation on our estimation. As the sample size increases, the random variation decreases, and the estimation accuracy increases.
In the above polling example, the pollsters may survey only a few people from a tiny number of polling stations. The variation thus obtained is more likely to be random than indicative of the entire population.
Don’t they sound the same?
It is easy to confuse between isolated evidence and random variation.
We can even say that isolated evidence is a special case of random variation.
However, there is one key difference.
Isolated evidence is rooted in the form of personal bias. Whereas the random variation comes purely from inadequate sample data.
That is why isolated evidence is more prevalent in people’s personal experiences. So, a friend asking us not to purchase a product is a case of isolated evidence. The star rating of the product on an e-commerce platform is statistical evidence and solves this problem if the rating is given by thousands of unknown people.
So, the next time we are tempted to make a conclusion based on a small sample size, check whether it is statistical evidence, or a case of isolated evidence or random variation.
Related Posts:
- Most Popular Perspectives from 2015
- Two Finals, Two Ties, and the Common Winner
- Fields Medal, Open Problem, and Business Decisions
<– How can we make difficult decisions?
Is analytics all hype and no substance? –>
If you liked reading this article, then please subscribe to our blog – Veracles. That way, you can receive interesting insights in email.
Also, please do follow Veravizion on LinkedIn, Twitter or Facebook to receive easy updates.