Model Interpretability

It is almost trite at this point for anyone to espouse the potential of machine learning in the medical field. There are a plethora of examples to support this claim — one would be Microsoft’s use of medical imaging data in helping clinicians and radiologists make an accurate cancer diagnosis. Simultaneously, the development of sophisticated AI algorithms has drastically improved the accuracy of such diagnoses. Undoubtedly, such amazing applications of medical data and one has all the good reasons to be excited about its benefits.

However, such cutting-edge algorithms are black boxes that might be difficult, if not impossible, to…

I recently stumbled across Symbolic Connection, a data podcast ran by experienced data practitioners, Thu Ya Kyaw and Koo Ping Shung. This podcast caught my attention as the guests are mainly from Singapore, one of the burgeoning tech hub of South-East Asia (it was also from where I am based.)

As a data analyst in a tech company, I am always curious about the work of analysts in other tech firms. So I listened to the podcast featuring Michael Ng from Agilent.

In this podcast, Michael shared about how his role as a data analyst manager at Agilent, advice for…

Recently, I graduated from Chemical Engineering and got in my first role as a data analyst in a tech company. Since then, as I spoke to students from my school about the move, many expressed the same interest and the same question…

‘How did you move from engineering to data science?’

That was the exact question I ask myself — how can I make the move? That same thought bugged for and pushed to start pursuing the skills of a data scientist a little over a year ago. …

I recently stumbled across Symbolic Connection, a data podcast ran by experienced data practitioners, Thu Ya Kyaw and Koo Ping Shung. This podcast caught my attention as the guests are mainly from Singapore, one of the burgeoning tech hub of South-East Asia (it was also from where I am based.)

As a data analyst in a tech company, I am always curious about the work of analysts in other tech firms. So I listened to the podcast featuring Cliff Chew, a senior data analyst at Grab.

In this podcast, Cliff shared about how his role as a data analyst in…

Conditional probability is one of the fundamental concepts in probability and statistics, and by extension, data science and machine learning.

In fact, we can think about the performance of a machine learning model using confusion matrix, which can be interpreted using a conditional probability perspective.

In this blog post, we will cover all that and more—

1. Introduction
2. A formal definition of conditional probability
3. Intuition to conditional probability: Moving to a new universe
4. Conditional Probability in Machine Learning and Confusion Matrix
5. How to learn more probability

Let’s get started.

Conditional Probability Introduction

Conditional probability is a method to reason about the outcome of an…

OPINION

“A proud man is always looking down on things … as long as you are looking down, you cannot see something that is above you.” — C.S. Lewis

Humans are prideful.

We think that human intelligence is the pinnacle, and thus mock artificial intelligence, even the face of its incredible intellect and creativity.

AI can code in any language, make amazing art (we’ll get into that), and drive better than us with Tesla’s Full Self-Driving. These are all advancements of the last year.

To think that AI won’t eventually do your job better than you is sheer pride, or the…

Probability and statistics — which comes first?

We almost always discuss probability and statistics hand-in-hand.

In fact, it is not uncommon for students to not know the distinction between statistics and probability.

But which comes first? What are the differences even?

It’s a chicken-and-egg problem. You can’t have probability without statistics, and vice versa. What do I mean by that? I’ll explain more in this post.

In this short post, I will highlight some of these differences between probability and statistics and some applications of probability and statistics in data science.

From this, I hope you will gain or have a new perspective of why data scientists…

Probability is not the easiest topic to master as its theoretical nuances can be esoteric and arguably obscure. I found a lot of resources online difficult to understand or lacking in rigour — but it need not be that way.

This is why I started this series to explain probability intuitively with easy examples. It’s bite-size data science that you can learn now in 5 minutes.

In this post, I will be using my favourite TV couple, Michael and Jan, as examples. Stay tuned.

This series will be in cover—

1. Probability Models and Axioms (you’re here!)
2. Probability vs Statistics
3. Conditional…