1. Logic Probability and Uncertainty

Deductive Logic and Plausible Reasoning

A is true ⟹ B is true

If we know $A$ is true then when can deduce that $B$ is true. If $B$ is not true we can deduce $A$ is not true.

Pasted image 20250506122112.webp|292

$A$ and $B$ both true is the region $A \cap B$ . $A$ or $B$ are true is the region $A \cup B$ .

If $A$ is not true we cannot deduce anything about $B$ since $B ∖ A$ is nonempty.

The same thing with $B$ is true, we can't deduce anything about $A$ though it increases the plausibility that $A$ is true. When changing the plausibility of one proposition based on another proposition we are making an induction.

Desired Properties of Plausibility Measures:

Degrees of plausibility are nonnegative
They qualitatively agree with common sense. Larger means greater plausibility
If a proposition can be represented in multiple ways, then they must have the same plausibility
Always take all relevant evidence into account
Equivalent states of knowledge are always given the same plausibility

Probability

Random Experiment - Outcome is not completely predictable. Can be repeated under the same conditions.
Outcome - the result of one single trial of the single trial of the random experiment
Sample Space - set of all possible outcomes of a single trial denoted $Ω$ or $U$ for universe.
Event - any set of possible outcomes of a random experiment
union - either event $A$ or event $B$
intersection - both event $A$ and event $B$
complement - set of outcomes not in $A$ denoted here as $\tilde{A}$
mutually exclusive - $A \cap B = \emptyset$ (disjoint)
partition - $⋃ B_{i} = Ω$ and $B_{i} \cap B_{j} = \emptyset$ , $i \neq j$

Axioms of Probability

$P (A) \geq 0$ for any event $A$
$P (Ω) = 1$
If $A$ and $B$ are mutually exclusive then $P (A \cup B) = P (A) + P (B)$

basic probability theorems

Basic Probability Theorems

$P (Ω) = B$ which is the powerset of $Ω$

1.3.1

$\begin{aligned} A \in B, P (A) = 1 - P (A^{c}) \end{aligned}$
pf:
$Ω = A \cup A^{c}, A \cup A^{c} = \emptyset$
$1 = P (Ω) = P (A \cup A^{c}) = P (A) + P (A^{c})$

1.3.2

$P (\emptyset) = 0$
pf:
$P (\emptyset) = 1 - P (\emptyset^{c}) = 1 - P (Ω)$

1.3.3

$A \subseteq B ⟹ P (A) \leq P (B)$
pf:
$B = A \cup (B \cap A^{c})$ and $A \cap (B \cap A^{c}) = \emptyset$

$P (B) = P (A \cup (B \cap A^{c})) = P (A) + P (B \cap A^{c})$

1.3.4

$A \in B, 0 \leq P (A) \leq 1$

pf:
$\emptyset \subseteq A \subseteq Ω ⟹ P (A) \leq P (Ω) and P (\emptyset) \leq P (A)$

1.3.5 Inclusion Exclusion

$A, B \in Ω ⟹ P (A \cup B) = P (A) + P (B) - P (A \cap B)$
pf:
$P (B) = P (A \cap B) + P (A^{c} \cap B)$
$A \cup B = A \cup (A^{c} \cap B) ⟹ P (A \cup B) = P (A) + P (A^{c} \cap B) = P (A) + P (B) - P (A \cap B)$

Continuity Theorem

$lim_{n \to \infty} P (A_{n}) = P (lim_{n \to \infty} A_{n})$

1.3.6

Let ${A_{n}}$ be non-decreasing
$lim_{n \to \infty} (A_{n}) = P (lim_{n \to \infty} A_{n}) = P (\cup_{n = 1}^{\infty} A_{n})$
Let ${A_{n}}$ be non-increasing
$lim_{n \to \infty} P (A_{n}) = P (lim_{n \to \infty} A_{n}) = P (\cap_{n = 1}^{\infty} A_{n})$
pf (non-decreasing):
$\begin{aligned} R_{1} = A_{1} \\ R_{2} = A_{2} - A_{1} \\ R_{n} = A_{n} - A_{n - 1} \\ P (lim_{n \to \infty} A_{n}) = P (\cup_{n = 1}^{\infty} A_{n}) = P (\cup_{n = 1}^{\infty} R_{n}) = \sum_{n = 1}^{\infty} P (R_{n}) \\ = lim_{k \to \infty} \sum_{n = 1}^{k} P (R_{n}) = lim_{k \to \infty} {P (A_{1}) + \sum_{n = 2}^{k} [P (A_{n}) - P (A_{n - 1})]} \\ = lim_{n \to \infty} P (A_{n}) \end{aligned}$

1.3.7 Boole's Inequality

Let ${A_{n}}$ be an arbitrary sequence of events

$P (\cup_{n = 1}^{\infty} A_{n}) \leq \sum_{n = 1}^{\infty} P (A_{n})$
pf:

$\begin{aligned} {\begin{cases} D_{n} = \cup_{i = 1}^{n} A_{i} ⟹ D_{n} is non-decreasing \\ D_{j} = D_{j - 1} \cup A_{j} ⟹ P (D_{j}) \leq P (D_{j - 1}) + P (A_{j}) \\ P (D_{j}) - P (D_{j - 1}) \leq P (A_{j}) \end{cases} \\ P (\cup_{n = 1}^{\infty} A_{n}) = P (\cup_{n = 1}^{\infty} D_{n}) = lim_{n \to \infty} {P (D_{1}) + \sum_{j = 1}^{n} [P (D_{j}) - P (D_{j - 1})]} \\ \leq lim_{n \to \infty} {P (A_{1}) + \sum_{j = 2}^{n} P (A_{j})} = lim_{n \to \infty} \sum_{j = 1}^{n} P (A_{j}) = \sum_{n = 1}^{\infty} P (A_{n}) \end{aligned}$

Marginal Probability - $P (A) = P (A \cap B) + P (A \cap B^{c})$

Joint Probability and Independence

Conditional probability of event $B$ relative to the hypothesis of event $A$

$\begin{matrix} P (A | A) = 1 and P (B | A) = P (A \cap B | A) \\ \frac{P (A \cap B | A)}{P (A | A)} = \frac{P (A \cap B)}{P (A)} \end{matrix}$

Conditional Probability

Let $A$ and $B$ be events where $P (A) > 0$

Pasted image 20250516160854.webp|230
$\begin{aligned} P (B | A) = \frac{P (A \cap B)}{P (A)} \\ where: \\ 1. P (B | A) \geq 0 \\ 2. P (A | A) = 1 \\ 3. P (\cup_{n = 1}^{\infty} B_{n} | A) = \sum_{n = 1}^{\infty} P (B_{n} | A) & B_{1}, B_{2}, \dots are mutually exclusive \end{aligned}$

Multiplication Rules

$\begin{aligned} P (A \cap B) = P (B | A) P (A) \\ P (A \cap B) = P (B \cap A) = P (A | B) P (B) \\ P (A \cap B \cap C) = P ((A \cap B) \cap C) = P (C | A \cap B) P (A \cap B) = P (C | A \cap B) P (B | A) P (A) \end{aligned}$

Total Law of Probability

$P (B) = \sum_{i = 1}^{k} P (A_{i}) P (B | A_{i})$
where $A_{i}, i = 1, 2, 3, \dots, k$ forms a partition of $Ω$

pf.
$\begin{aligned} P [B \cap Ω] \\ = P [B \cap (A_{1} \cup A_{2} \cup \dots \cup A_{k})] \\ = P [(B \cap A_{1}) \cup \dots \cup (B \cap A_{k})] \\ = P [(B | A_{1})] + P [B | A_{2}] + \dots + P [B | A_{k}] \end{aligned}$

Bayes Theorem

Assume $A_{1}, A_{2}, \dots, A_{k}$ be events such that $P (A_{i}) > 0$ and they form a partition.
For any event $B$ :
$P (A_{j} | B) = \frac{P (A_{j}) P (B | A_{j})}{\sum_{i = 1}^{k} P (A_{i}) P (B | A_{i})}$
pf:
$P (A_{j} | B) = \frac{P (B \cap A_{j})}{P (B)} = \frac{P (A_{j} \cap B)}{P (B)} = \frac{P (A_{j}) P (B | A_{j})}{P (B)}$

Independence

$A$ and $B$ are independent if:
$P (A \cap B) = P (A) P (B)$
Equivalently if:
$P (B | A) = P (B)$
Mutually Independent iff they are pairwise independent:

$\begin{matrix} P (A_{i} \cap A_{j}) = P (A_{i}) P (A_{j}) \\ and for any collection \\ P (A_{d_{1}} \cap A_{d_{2}} \cap \dots \cap A_{d_{k}}) = P (A_{d_{1}}) P (A_{d_{2}}) \dots P (A_{d_{k}}) \end{matrix}$

Baye's Theorem

Let $B_{1}, B_{2}, \dots, B_{n}$ be a set of unobservable events which partition the universe.

We start with our prior probability $P (B_{i})$ for each which can be informative of non-informative. This distribution gives the weighting of our prior beliefs. Then we find that $A$ has occurred.

The Likelihood of the unobservable events $B_{1}, \dots, B_{n}$ is the conditional probability that $A$ has occurred given each $B_{i}$ , $P (A | B_{i})$ . It's the weight given to each of the $B_{i}$ events given that $A$ occurred.

The posterior probability is $P (B_{i} | A)$ , this distribution contains the weight we attach to each of the evens $B_{i}$ after we know event $A$ has occurred.

Bayesian Universe

Has two dimensions, one observable and one unobservable where the observable dimension is the horizontal.

Pasted image 20250516171305.webp|219

Pasted image 20250516171330.webp|220

Odds

o d d s (A) = \frac{P (A)}{P (A^{c})}

Bayes Factor

\begin{array}{r} B \times p r i o r o d d s = p o s t e r i o r o d d s \\ B = \frac{P (D | C)}{P (D | C^{c})} \end{array}

$D =$ data that occurred, $C$ = outcome in question