Concepts in Probability

Nima Hejazi
nhejazi@hsph.harvard.edu

Harvard Biostatistics

September 3, 2025

Basic Concepts of Probability

Intro to Probability

People often colloquially refer to probability…

  • “What are the chances the Red Sox will win this weekend?”
  • “What’s the chance of rain tomorrow?”
  • “What is the chance that a patient responds to a new therapy?”

Formalizing concepts and terminology around probability theory is essential for better understanding probability (and statistics).

Random Experiments

A random experiment is an action or process that leads to one of several possible outcomes.

  • For example, flipping a coin leads to two possible outcomes: either heads or tails.

The probability of an outcome is the proportion of times that the outcome would occur if the random phenomenon could be observed an infinite number of times.

  • If a fair coin is flipped an infinite number of times, heads would be obtained 50% of the time.

Outcomes and events

An outcome in a study is the result observable once the experiment has been conducted.

  • The sum of the faces on two dice that have been rolled.
  • The response of a patient treated with an experimental therapy.

An event is a collection of outcomes.

  • The sum after rolling two dice is 7.
  • 22 of 30 patients in a study have a good response to a therapy.

Events can be referred to by letters. For example, if A is the event of rolling a number smaller than 3 on a die, then A={1,2}.

Disjoint (Mutually Exclusive) Events

Two events or outcomes are called disjoint or mutually exclusive if they cannot both happen at the same time.

Here, A and B being disjoint means

  • P(A∪B)=P(A)+P(B)=2/6+2/6=4/6
  • The probability of rolling a 1, 2, 4, or 6 on a six-sided die is 4/6.

Addition Rule for Disjoint Events

If A and B are two disjoint events, then the probability that either occurs is P(A∪B)=P(A)+P(B)1.

If there are k disjoint events A1,…,Ak, the probability that one of these outcomes will occur is P(A1)+P(A2)+⋯+P(Ak)

  1. The ∪ symbol denotes the union of two events; i.e., P(A or B). Statistics uses the inclusive “or,” such that A or B occurring means A, B, or both A and B occur.

General Addition Rule

Suppose that we are interested in the probability of drawing a diamond or a face card out of a standard 52-card deck.

Does P(diamond or face card)=13/52+12/52?

General Addition Rule…

No, we need to correct the double counting of the three cards that are in both events, subtract the probability that both events occur… P(diamond or face)=P(diamond)+P(face)−P(diamond and face)=13/52+12/52−3/52=22/52

Thus, for any two events A and B, the probability that either occurs is P(A∪B)=P(A)+P(B)−P(A∩B)1.

  1. The ∩ symbol denotes the intersection of two events; i.e., P(A and B).

Sample Space

A sample space is an exhaustive list of mutually exclusive outcomes.

Suppose the possible k outcomes are denoted O1,O2,…,Ok. The sample space can be expressed as S={O1,O2,…,Ok}.

Given a sample space S={O1,O2,…,Ok}, the sum of the probabilities of each outcome must equal 1 – that is, ∑i=1kP(Oi)=1

Complement of an Event

Let D={2,3} represent the event that the outcome of a single die roll is 2 or 3.

The complement of D represents all possible outcomes within the sample space that are not in D.

Complement of an Event…

The complement of an event A is denoted by AC.

An event and its complement are mathematically related:

P(A)+P(AC)=1P(A)=1−P(AC)

Independent Events

Two events A and B are independent if the probability that both A and B is the product of their probabilities: P(A∩B)=P(A)P(B)

A blue die and a green die are rolled. What is the probability of rolling two 1’s?

Conditional Probability

An example from childhood mortality

Published in Patel, et al., NEJM (2015) Vol 372, pp 331 - 340.

Conditional Probability: Intuition

Consider height in the US population.

What is the probability that a randomly selected individual in the population is taller than 6 feet, 4 inches?

  • Suppose you learn that the selected individual is a professional basketball player.
  • Does this change the probability that the individual is taller than 6 feet, 4 inches? Yes or no, and why?

Conditional Probability: Concept

The conditional probability of an event A, given a second event B, is the probability of A happening, knowing that B has happened. This conditional probability is denoted P(A∣B).

Toss a fair coin three times. Let A be the event that exactly two heads occur, and B the event that at least two heads occur.

  • Conditioning on B means that the sample space consists of {HHH,HHT,HTH,THH} – all possible sets of three tosses where at least two heads occurred.
  • In this restricted set of outcomes, A, consists of the last three, so P(A∣B)=3/4.

Conditional Probability: Formal Definition

As long as P(B)>0, then P(A∣B)=P(A∩B)P(B).

From the definition, P(A∣B)=P(at least two heads and exactly twoheads)P(at least two heads)=P(exactly twoheads)P(at least two heads)=3/84/8=3/4

Independence, Again…

A consequence of the definition of conditional probability:

  • If P(A∣B)=P(A), then A and B are independent; knowing B offers no information about whether A occurred.

Thus, independence means that conditioning has no effect since the two event spaces do not overlap.

General Multiplication Rule

If A and B are two events, then P(A∩B)=P(A∣B)P(B).

Rearranging the definition of conditional probability yields this P(A∣B)=P(A∩B)P(B)→P(A∣B)P(B)=P(A∩B)

Unlike the previously mentioned multiplication rule, this is valid for events that might not be independent.

Positive Predictive Value and Bayes’ Theorem

Pre-natal Testing for Trisomy 21, 13, and 18

Some congenital disorders are caused by an additional copy of a chromosome being attached to another in reproduction.

  • Trisomy 21: Down syndrome, approximately 1 in 800 births
  • Trisomy 13: Patau’s syndrome, physical and mental disabilities, approximately 1 in 16,000 newborns
  • Trisomy 18: Edward’s syndrome, nearly always fatal, either in stillbirth or infant mortality, occurs in about 1 in 6,000 births

Cell-free fetal DNA (cfDNA), copies of embryo DNA present in maternal blood, can be used as a non-invasive test.

Cell-free Fetal DNA-based Testing

Initial testing of the technology was done using archived samples of genetic material in children whose trisomy status was known.

The results are variable, but generally very good:

  • Of 1000 unborn children with one of the disorders, about 980 have cfDNA that tests positive. The test has high sensitivity (true positive rate).
  • Of 1000 unborn children without the disorders, about 995 test negative. The test has high specificity (true negative rate).

Cell-free Fetal DNA-based Testing…

The designers of a diagnostic test strive for accuracy: A test should have high sensitivity and specificity.

A family with an unborn child undergoing testing wants to know the likelihood of the condition being present if the test is positive.

Suppose a child has tested positive for trisomy 21. What is the probability the child does have trisomy 21, given the positive test result?

Defining Events in Diagnostic Testing

Events of interest in diagnostic testing:

  • D = {disease present}
  • DC = {disease absent}
  • T+ = {positive test result}
  • T− = {negative test result}

Could use T and TC, but T+ and T− are consistent with notation in medical and public health literature.

Characteristics of a Diagnostic Test

The following measures are all characteristics of a diagnostic test.

  • Sensitivity = P(T+∣D)
  • Specificity = P(T−∣DC)
  • False negative rate = P(T−∣D)
    • Note that P(T−∣D)=1−P(T+∣D), i.e., 1 - sensitivity
  • False positive rate = P(T+∣DC)
    • Note that P(T+∣DC)=1−P(T−∣DC), that is, 1 - specificity

Positive Predictive Value of a Test

Suppose an individual tests positive for a disease.

The positive predictive value (PPV) of a diagnostic test is the probability that the disease is present, given the test returns a positive results: PPV = P(D∣T+)

The characteristics of a diagnostic test include P(T+∣D), among other probabilities, but not the reverse conditional P(D∣T+).

Bayes’ Theorem (or Bayes’ Rule)

Bayes’ Theorem (simplest form): P(A∣B)=P(B∣A)P(A)P(B)

Follows directly from the definition of conditional probability, noting that P(A)P(B∣A) equals P(A and B):

P(A∣B)=P(A∩B)P(B)=P(B∣A)P(A)P(B)

The Denominator P(B) in Bayes’ Theorem

Bayes’ Theorem is usually stated differently, since, in many problems, P(B) is not given directly but is calculated via a general multiplication rule:

Suppose A and B are events. Then, P(B)=P(B∩A)+P(B∩AC)=P(B∣A)P(A)+P(B∣AC)P(AC)

Bayes’ Theorem can be written as: P(A∣B)=P(A)P(B∣A)P(B)=P(B∣A)P(A)P(B∣A)P(A)+P(B∣AC)P(AC)

Bayes’ Theorem for Diagnostic Tests

P(D∣T+)=P(D∩T+)P(T+)=P(D∩T+)P(D∩T+)+P(DC∩T+)=P(T+∣D)P(D)P(T+∣D)P(D)+P(T+∣DC)P(DC)=sensitivity×prevalence[sensitivity×prevalence]+[(1 - specificity)×(1 -prevalence)]

Bayes’ Theorem for Diagnostic Tests…

P(D∣T+)=P(D∩T+)P(T+)=P(D∩T+)P(D∩T+)+P(DC∩T+)

HST 190: Introduction to Biostatistics

Concepts in Probability Nima Hejazi nhejazi@hsph.harvard.edu Harvard Biostatistics September 3, 2025

  1. Slides

  2. Tools

  3. Close
  • Concepts in Probability
  • Basic Concepts of Probability
  • Intro to Probability
  • Random Experiments
  • Outcomes and events
  • Disjoint (Mutually Exclusive) Events
  • Addition Rule for Disjoint Events
  • General Addition Rule
  • General Addition Rule…
  • Sample Space
  • Complement of an Event
  • Complement of an Event…
  • Independent Events
  • Conditional Probability
  • An example from childhood mortality
  • Conditional Probability: Intuition
  • Conditional Probability: Concept
  • Conditional Probability: Formal Definition
  • Independence, Again…
  • General Multiplication Rule
  • Positive Predictive Value and Bayes’ Theorem
  • Pre-natal Testing for Trisomy 21, 13, and 18
  • Cell-free Fetal DNA-based Testing
  • Cell-free Fetal DNA-based Testing…
  • Defining Events in Diagnostic Testing
  • Characteristics of a Diagnostic Test
  • Positive Predictive Value of a Test
  • Bayes’ Theorem (or Bayes’ Rule)
  • The Denominator P(B) in Bayes’ Theorem
  • Bayes’ Theorem for Diagnostic Tests
  • Bayes’ Theorem for Diagnostic Tests…
  • f Fullscreen
  • s Speaker View
  • o Slide Overview
  • e PDF Export Mode
  • r Scroll View Mode
  • b Toggle Chalkboard
  • c Toggle Notes Canvas
  • d Download Drawings
  • ? Keyboard Help