Conditional Probability Explained

Have you ever wondered what the likelihood of getting your dream job or winning the lottery is? At first glance, the idea seems completely out of reach, because these instances are influenced by many different factors. This is where probability comes in. While it may not help you win the lottery, it helps you calculate the odds for a variety of different scenarios – in fact, you’ve probably used it multiple times just today.

In less casual terms, probability is used by many industry professionals for its ability to resolve complex real-world problems in a more quantifiable way. Whether it’s through the different types of distribution, expected values, or mathematical modeling methods like the Monte Carlo simulation, mastering this theory will help you make a giant leap towards a successful data science career.

In this tutorial, we will introduce you to the key concept of conditional probability and how it applies to real-life instances. You will also learn how to:

Distinguish between independent and dependent events
Interpret the conditional probability formula
Apply the law of total probability

What Is Conditional Probability?

Essentially, conditional probability is the likelihood of an event occurring, assuming a different one has already happened. Otherwise said, there must be some sort of relationship with the past. Moreover, its formula, which we will expand on in this tutorial, is based on the Bayes’ Theorem.

But first, in order to fully understand how to calculate conditional probability, we must look at the events that can affect it.

What Are Independent and Dependent Events?

There are two types of events that can influence conditional probability:

Independent
Dependent

It’s important to know the differences in order to successfully solve a problem. In fact, we use conditional probability to distinguish between the events.

What Is an Independent Event?

We call the theoretical probability that remains unaffected by other occurrences an independent event.

The easiest example is a coin flip – you always have a 50% chance of landing on your desired side, regardless of the result of the previous throw. In this case, we have two events – A and B. If A is flipping heads and B represents getting the same on the previous try, the probability doesn’t change – it remains 0.5. Therefore, we say that:

$𝑃 (𝐴) = 𝑃 (𝐴 | 𝐵) = 0.5$

What this means is that the probability of the coin landing on heads in a new flip is unconditional with respect to previous flips.

In cases where any two events are independent, the probability of their intersection is the product of the individual probabilities:

$𝑃 (𝐴 \cap 𝐵) = 𝑃 (𝐴) \times 𝑃 (𝐵)$

What Is a Dependent Event?

The probabilities of dependent events vary as conditions change.

For instance, what is the probability of drawing the Queen of Spades? Normally, we have exactly one favorable outcome and 52 elements in the sample space, so the result is:

$𝑃 (𝐴) = \frac{1}{52}$

Now, imagine we know that we drew a spade. The new sample space contains all the 13 cards from the suit only and the probability becomes:

$𝑃 (𝐴 | 𝐵) = \frac{1}{13}$

The two events are, therefore, dependent.

Alternatively, our sample can consist of only four cards if we know that the one we have drawn is a queen instead of a spade. Thus, the probability of drawing the Queen of Spades becomes:

$𝑃 (𝐴 | 𝐵) = \frac{1}{4}$

That is because the probability of getting our desired card alters if we know it is a queen. In other words, A and C are also dependent.

With this example, you could clearly see how the probability of an event changes depending on the information we have.

The Conditional Probability Formula

By definition, the conditional probability equals the probability of the intersection of events A and B over the probability of event B occurring:

$𝑃 (𝐴 | 𝐵) = \frac{𝑃 (𝐴 \cap 𝐵)}{𝑃 (𝐵)}$

This holds true only if the probability of $B > 0$ . This is logically so because if we have $P (B) = 0$ , then the event would never occur. Thus, $P (A | B)$ would not be interpretable.

First, to satisfy the conditional probability formula, we need both events B and A to occur simultaneously. This suggests that the intersection of A and B would consist of all our favorable outcomes.

Second, the conditional probability requires that event B occurs, so the sample space would simply be all outcomes where event B is satisfied.

It’s important to remember that the order in which we write the elements for the conditional probability is crucial. $P (A | B)$ is not the same as $P (B | A)$ , even if the numeric values are equal.

To illustrate, let’s explore the characteristics of Hamilton College’s class of 2018:

5% percent of the students who got a degree in Economics graduated with honors
At the same time, 5% of all students who graduated with honors completed a concentration in Economics

These two statements might have the same conditional probability, but they hold completely different meanings.

In particular, the first suggests that only four of the 80 Economics majors graduated with a distinction:

$𝑃 (𝐻 | 𝐸) = \frac{4}{80}$

Meanwhile, the second statement shows that four out of the 80 students who graduated with high grades completed a degree in Economics:

$𝑃 (𝐸 | 𝐻) = \frac{4}{80}$

Conditional Probability in Real Life: An Example

Many scientific papers rely on conducting experiments or surveys. They often provide summarized statistics we use to analyze and interpret how certain factors affect one another.

Imagine you conducted a survey where you asked 100 men and women of all ages if they eat meat and obtain the following results:

	Vegetarian	Not Vegetarian	Total
Women	15	32	47
Men	29	24	53
Total	44	56	100

We see 15 of the 47 women that participated are vegetarian, as well as 29 out of the 53 men.

Now, if A represents being vegetarian and B represents being a woman, then $P (A | B)$ and $P (B | A)$ express different events.

The likelihood of a woman being vegetarian is:

$𝑃 (𝐴 | 𝐵) = \frac{15}{47}$

Meanwhile, the likelihood of a vegetarian being a woman is:

$𝑃 (𝐵 | 𝐴) = \frac{15}{44}$

Based on these outcomes, we can conclude that it is more likely for a vegetarian to be female, than for a woman not to eat meat. This goes to show that in probability theory things are never straightforward.

The Law of Total Probability

Now that you understand the distinctions between the different conditional probabilities of two events, we can introduce an important concept – the law of total probability.

Let’s say A is the union of some infinitely many events:

$𝐴 = 𝐵_{1} \cup 𝐵_{2} \cup . . . \cup 𝐵_{𝑛}$

This law dictates that the probability of A is:

$𝑃 (𝐴) = 𝑃 (𝐴 | 𝐵_{1}) \times 𝑃 (𝐵_{1}) + 𝑃 (𝐴 | 𝐵_{2}) \times 𝑃 (𝐵_{2}) . . .$

So, going back to our survey example, the probability of being vegetarian equals

$𝑃 (𝑣 𝑒 𝑔 𝑒 𝑡 𝑎 𝑟 𝑖 𝑎 𝑛) = 𝑃 (𝑣 𝑒 𝑔 𝑒 𝑡 𝑎 𝑟 𝑖 𝑎 𝑛 | 𝑚 𝑎 𝑙 𝑒) \times 𝑃 (𝑚 𝑎 𝑙 𝑒) + 𝑃 (𝑣 𝑒 𝑔 𝑒 𝑡 𝑎 𝑟 𝑖 𝑎 𝑛 | 𝑓 𝑒 𝑚 𝑎 𝑙 𝑒) \times 𝑃 (𝑓 𝑒 𝑚 𝑎 𝑙 𝑒)$

If we plug in values from the results table, we get that the probability of being vegetarian equals:

$\frac{29}{53} \times \frac{53}{100} + \frac{15}{47} \times \frac{47}{100} = 0.44$

Therefore, according to the survey, there is a 44% chance of someone being vegetarian.

Conditional Probability: Next Steps

As you can see, there are many benefits to learning how to apply probability in order to solve real-life problems. In fact, the theory is used in branches such as finance, business analytics, healthcare, and many more. In other words, it is indeed an essential skill for everyone looking to work with data.

Are you ready for the next step toward a career in data science?

The 365 Data Science Program offers self-paced courses led by renowned industry experts. Starting from the very basics all the way to advanced specialization, you will learn by doing with a myriad of practical exercises and real-world business cases. If you want to see how the training works, start with our free lessons by signing up below.