2: Axioms of probability

Section 1: Basic Probability Concepts - 2: Axioms of Probability

Probability theory requires a rigorous mathematical foundation to avoid paradoxes and ensure consistency. This foundation is provided by the Kolmogorov Axioms, three fundamental rules that define how probability must behave for any event within a sample space ($S$).

1. Non-Negativity: For any event $E \subseteq S$, the probability of $E$ is a non-negative real number:

   $$P(E) \geq 0$$

   This axiom states probabilities cannot be negative. For example, the probability of rolling an even number on a fair die cannot be -0.5.

2. Normalization: The probability assigned to the entire sample space (the event that something happens) is exactly 1:

   $$P(S) = 1$$

   This establishes probability as a measure of certainty on a scale from 0 (impossible) to 1 (certain). For a coin toss, $P(\{\text{Heads}, \text{Tails}\}) = 1$.

3. Countable Additivity: For any sequence of mutually exclusive (disjoint) events $E_1, E_2, E_3, \ldots$ (where $E_i \cap E_j = \emptyset$ for $i \neq j$), the probability of their union is the sum of their individual probabilities:

   $$P\left(\bigcup_{i=1}^{\infty} E_i\right) = \sum_{i=1}^{\infty} P(E_i)$$

   This is the core rule for combining probabilities of events that cannot occur simultaneously. For a single die roll, events $E_1 = \{1\}$ and $E_2 = \{2\}$ are disjoint. The probability of rolling a 1 or a 2 is $P(\{1\} \cup \{2\}) = P(\{1\}) + P(\{2\}) = \frac{1}{6} + \frac{1}{6} = \frac{1}{3}$.

Key Implications:

• Probability of the Empty Set: The probability of an impossible event (the empty set $\emptyset$) is 0: $P(\emptyset) = 0$. This follows from additivity (consider $S \cup \emptyset \cup \emptyset \cup \ldots$) and normalization ($P(S) = 1$).
• Finite Additivity: For a finite collection of mutually exclusive events $E_1, E_2, \ldots, E_n$, additivity holds: $$P\left(\bigcup_{i=1}^{n} E_i\right) = \sum_{i=1}^{n} P(E_i).$$
• Probability Bounds: For any event $E$, $0 \leq P(E) \leq 1$. This follows from non-negativity, normalization, and the fact that $E \subseteq S$.
• Complement Rule: $P(E^c) = 1 - P(E)$, since $E$ and $E^c$ are disjoint and their union is $S$ (use finite additivity and normalization).

These axioms are minimal but powerful. All other probability rules (like the inclusion-exclusion principle, conditional probability formulas, or Bayes' theorem) are derived from these three core principles. They provide the consistent framework upon which the entire edifice of probability theory is built, ensuring logical coherence for analyzing random phenomena.