Author of this article: Andreas Soller

Risk factors reading data

This article discusses the risk associated with evaluating metrics, emphasizing the importance of maintaining an open mind and being aware of biases.

Reading time of this article:

1 min read (264 words)

Publishing date of this article:

Nov 11, 2023 – Updated Dec 26, 2024 at 09:15

This article is tagged as:

On this page

Stay open minded

Even though we have set up our metrics in the best way we could foresee, we might not track the right data. Therefore it is important to stay open minded (sceptical) about your data. It might be that we didn’t target the correct user group as the changes in working software have a greater impact on another group we didn’t expect. Open minded refers to the fact that you don’t just blindly trust your metrics but check the metrics once more against your data.

Be aware of biases

“Bias: a strong feeling in favour of or against one group of people, or one side in an argument, often not based on fair judgement.” – Oxford Dictionary

Think about idea bias: we perceive our own ideas as better than those of others. Statistics show that up to 80% of all ideas are not awesome for users. Sometimes what we consider as great ideas are even making things worse for our users. This means that 8 out of 10 ideas will statistically not bring the expected value for users and might even result in sunk cost.

There are many types of biases such as

  • Statistical bias
  • Bias of self reporting data,
  • Cognitive bias,
  • (…)

I recommend the following medium article for a deep dive into different types of biases: “Cognitive bias cheat sheet”

References and recommended readings

  • Seiden, Joshua (2019): Outcomes over Output. Why customer behavior is the key metric for business success. United States (On demand): Sense & Respond Press

How did you like this article?

Thank you for your feedback.