# Probability

• Random variable x set of outcomes $$S=\{x_1,x_2,\cdots,x_N\}$$ outcomes can be discrete or continuous. $$S_{coin}=\{h,t\}, S_{V_x}=\{V_x,|V_x|\lt\infty\}$$.
• Event is any subset of outcomes $$E\subset S$$ and is assigned a probability $$p(E)$$.
• I.e. $$p_{dice}(\{1\})=\frac{1}{6}$$, $$p_{dice}(\{1,3\})=\frac{1}{3}$$.
• Union of Events consider events $$A$$ ($$B$$) where sum of 2 dice is divisible by 3 (4). $$A\cup B$$, A or B, is the set of outcomes where the sum of 2 dice is divisible by either 3 or 4.
• Intersection of Events $$A\cap B=AB$$ (not sure about the last notation, but it was written in class), A and B, would be where the sum of 2 dice is divisible by both 3 and 4, i.e. divisible by 12.
• Disjoint of Events $$A\cap B=\varnothing$$ (disconnected)
• Complement $$E_c=S\setminus E$$

## Axioms

• $$\forall E\subseteq S.p(E)\geq 0$$
• $$p(S)=1$$
• $$p(A\cup B)=p(A)+p(B)$$ if $$A$$ and $$B$$ are disjoint events
• $$p(E_c)=1-p(E)$$
• Objective probabilities: $$p(A)=\lim_{N\to\infty}\frac{N_A}{N}$$ observation
• Frequentist $$\leftrightarrow$$ Bayesian statistics
• Subjective probabilities: theoretical estimates for the probabilities using a model
• Computing probabilities: $$S$$ discrete and finite, $$S=\{x_1,x_2,\cdots,x_N\}$$, assume $$p(\{x_i\})=\frac{1}{N}$$, $$1\lt i\lt N$$. Then, $$p(E)=\frac{\#\:outcomes\:in\:E}{\#\:outcomes\:in\:S}$$.

## Combinatorics

### Exercises

#### Question 1

How many distinct ways you can arrange the 24 letters of the alphabet? W=24!

N distinct objects can be arranged in $$W=N!$$ ways.

Letters in ’WHAT’: W=4!

#### Question 2

Letters in ’CHEESE’: W=6!/3! = Number of ways to arrange it/number of ways to arrange the non-distinct letters

#### Question 3

Letters in ’FREEZER’: W=7!/(3!2!1!1!).

3! from the Es, 2! from the Rs, 1! from the F, 1! from the Z.

Multinomial coefficient gives the number of arrangements W of N objectects from k distinct categories each of which appears $$N_j$$ times, where $$N=\sum_{j=1}^kN_j$$. $$W=\frac{N!}{N_1!\cdots N_{k}!}=\frac{N!}{\prod_j N_j!}$$.

We get a special case for $$k=2$$, the binomial coefficient. $$W=\frac{N!}{N_1!N_2!}=\frac{N!}{N_1!(N-N_1)!}=N$$ choose $$N_1$$. $$W=N\choose{M}$$

#### Question 4

For $$N$$ coin tosses, how many are tails?

$$N_A:=$$ Outcome $$A$$ in $$N$$ trials.

We want $$p_N(N_T)$$.

For event $$E$$: $$N_T=5$$, $$N=12$$. Then, $$W_E$$ is 12 choose 5. $$W_S=(2!)^{12}$$. $$P(5)=\frac{{12\choose 5}}{2^12}$$.

So, $$P(N_T)=\frac{W_E}{W_S}=\frac{\frac{N!}{N_T!(N-N_T)!}}{(2!)^N}=\frac{1}{(p_T)^{N_T}(1-p_T)^{N-N_T}}({N\choose N_T})$$.

## Binomial Distribution

2 outcomes with $$p_a,p_b=1-p_a$$ in $$N$$ trials, $$p_N(N_a)=(p_a)^{N_a}(1-p_a)^{N-N_a}{N\choose N_a}$$.

## Multinomial Distribution

$$N$$ trials with $$k$$ outcomes with probabilities $$p_1,\cdots,p_k$$. The probability of finding $$N_1,N_2,\cdots,N_k$$ outcomes $$j$$ with $$N=\sum_{j=1}^kN_j$$. $$p_N(N_1,N_2,\cdots,N_k)=N!\prod_{j=1}^k \frac{1}{N_j!}p_j^{N_j}$$.

### Stirling’s Approximation

$$ln N! = N\ln N - N$$

$$N! = \exp(N\ln N-N) = \frac{N^N}{\exp(N)}=\left(\frac{N}{e}\right)^{N}$$.

Stirling’s: $$N! = \sqrt{2\pi N}\left(\frac{N}{e}\right)^N$$.

How? $$N! = \int x^n\exp(-x)dx = \Gamma(n+1)$$.

$$(N+1/2)\ln(N+1/2) - (N+1/2) - (1/2\ln1/2-1/2) - (N\ln N - N)$$ $$(N+1/2)(\ln N + \ln(1+1/(2N))) - N-1/2 - 1/2\ln1/2+1/2 - N\ln N + N$$ $$1/2\ln N + (N+1/2)\ln(1+1/(2N)) - 1/2\ln1/2$$ $$\ln \sqrt{N} + ~0 - 1/2\ln 1/2$$ Thus, the correction is of the order $$\sqrt{N}$$, which is what Stirling’s formula has.

## Ways

$$W = \frac{N!}{\prod_{j=1}^kN_j!}$$, $$\sum_{j=1}^kN_j=N$$. $$p_j = N_j/N$$.

### Approximating

#### Ways

$$W \approx = \sqrt{2\pi N}\left(\frac{N}{e}\right)^N\frac{1}{\prod_i N_i!} = \frac{1}{\prod_j p_j^{N_j}}$$

$$\ln W = -\sum_j N_j\ln p_j$$

$$\frac{\ln W}{N} = -\sum_j p_j\ln p_j = \frac{S}{Nk_B}\Rightarrow S = k_B\ln W = -k_B\sum_j p_j\ln p_j$$. Thus, we recovered our typical entropy! This makes sense since Entropy is related to the number of ways a system can be arranged.

The entropy of a fair dice is: $$S = k_B\ln 6$$.

No knowledge implies fiar dice maximizes the entropy.

Lets say we have a maximally unfair dice $$p_6=1,p_{i\neq 6}=0$$. Then the entropy is minimized. Assume we have no missing information.

Lets say someone says, $$\langle N\rangle_{dice} = 3.5$$. Then we would assume it is a fair dice.

If someone says $$\langle N\rangle_{dice} = 3.0$$ we would assume it is an unfair dice (weighted lower). There is a distribution of probabilities for the individual dice probabilities to be:

• $$p_3 = 1$$, $$p_{i\neq 3} = 0$$
• $$p_2=p_4=\frac{1}{2}$$, $$p_{i\neq 2,4}=0$$

Thus, there is an increase in missing information.

Created: 2023-06-25 Sun 02:29

Validate