![[Screencast from 06-22-2024 09_19_19 PM (trimmed).gif|500]] # Probability Theory **Probability theory** is the branch of mathematics concerned with probability. A *probability* describes how likely an event is to occur; it is always between 0 and 1. It can be thought of as a *ratio of likelihood* for a specific event(s) to occur out of all the events possible. --- > [!tldr] **History of Probability Theory** > Probability theory was originally invented to *study gambling*. One model in particular, the [[Sampling Techniques#Monte Carlo Simulations|Monte Carlo Simulation]], is even named after a famous casino. | Term | Symbol | Definition | | ------------------- | :----: | -------------------------------------------------------------------- | | Experiment | | | | Sample Space | $S$ | The set of all possible outcomes of an experiment | | Event | $E$ | A subset of the sample space that represents some group of outcomes. | | Probability Measure | | A real-valued function defined on a set of | | | | | ## The Probability Space ### The Sample Space > [!abstract] **Definition:** Sample Space ($S$ or $\Omega$) > Contents > [!abstract] **Definition:** Event ($E$) > An event is some > > - When dealing with finite sample spaces, the event space is simply all subsets of the sample space (unless defined otherwise). $P(\text{Event})=\lim_{n \to \infty} \frac{\text{count(Event)}}{n}$ An experiment with more than one possible outcome is *random/stochastic*, while one with a single outcome is *deterministic*. --- The **probability measure** is a type of [[Measure Theory|measure]] a --- All of these concepts together allow us to formally model an *experiment* using a mathematical construct known as a **probability space** composed of: 1. *The Sample Space ($S$ or $\Omega$)* 2. *The Event Space ($\mathcal{F}$)* 3. *A Probability Function ($P$)* An outcome is the result of a single execution of the model. According to the law of large numbers (LLN), the empirical probability If $S$ is a finite nonempty sample space of equally likely outcomes, and $E$ is an event, that is, a subset of $S$, then the probability of $E$ is $p(E) = \frac{|E|}{|S|}$. The **sample space** of an experiment or random trial is *the set of all possible outcomes*. There are some additional conditions that must be met in order to be a true sample space: 1. The outcomes must be *mutually exclusive*. 2. The outcomes must be *collectively exhaustive*. - Meaning that for every trial, one outcome must take place. 3. ### The Event Space - [ ] event - [ ] experiment vs random variable - [ ] outcome (sample?) When discussing probabilities, there is always some implied context which can be formally described as an **experiment**. Experiments can be repeatedly infinitely In order to precisely speak about probability, we must first define two sets: *the sample space* and *the event space*. In probability theory, an **experiment** is any procedure that can be infinitely repeated as a well-defined set of possible outcomes known as the **sample space**, $S$. > [!info]+ **Definition:** Conditional Probability > Conditional probability states “what is the chance of an event $E$ happening give that I have already observed some other event $F$.” > > The probabili' > $P(A|B)=\frac{P(A \cap B)}{P(B)}$ #### Dependent vs Independent This concept was pretty counterintuitive to me at first. If you ignore their names, these two properties essentially describe whether or not one event influences the outcome of another It’s also important to distinguish the nature of a scenario - Sequential events can be both dependent (i.e. ) and independent (i.e. rolling an ) ## Core Probability Notations > [!hint] Probability Theory vs Set Theory > > It can often be helpful to think of probabilities in terms of [[set theory]]. > > The **sample space** is the *set of all possible outcomes*. > - Often denoted as $S$ or $\Omega$ > > The **event space** contains *subsets of the sample space* that represent *individual events*. If the sample space is finite, then the event space may simply be the set of all subsets of the sample space. The cardinality/magnitude The probability of an event $E$ ![[Pasted image 20240922232444.png|325]] ![[Pasted image 20240812125451.png|300]] ## Probability Identities - [ ] AND - [ ] Conditional Probability - [ ] OR ### The Axioms of Probability These are also known as the *Kolmogorov axioms* > [!summary] **Axioms of Probability** > **Axiom 1:** All probabilities are numbers between 0 and 1. > $0 \le \Pr(s) \le 1 \text{ for each } s \in S$ > > --- > > **Axiom 2:** All outcomes must be from the *sample space*, $s$. Therefore, the probability of picking an outcome in the sample space is always 100%. > $\sum_{s \in S} \Pr(s)=1$ > also written as $\Pr(\Omega)=1$. > > --- > **Axiom 3:** If two events $A$ and $B$ are mutually exclusive, then: > $P(A \cup B)=P(A)+P(B)$