必须知道的概率论知识

• 常见分布
• 期望
• 方差
• 性质

一维变量

离散随机变量

def

离散随机变量X的分布列
$$P=(X=x_i)=p(x_i)i=1,2,...,n$$
则X的数学期望为：${\sum }_{i=1}^n{x}_{i}p(x_i)$，记为;
$$E(X)=\sum _{i=1}^n{x}_{i}p(x_i)$$
若X的取值可列，且无穷级数${\sum }_{i=1}^{\infty }{x}_{i}p(x_i)$收敛，则$E(X)={\sum }_{i=1}^{\infty }{x}_{i}p(x_i)$.

常见分布

几何分布

$p(x)=P(X=x)=p(1-p{)}^{x-1},x=1,2,....$.

期望

$E(X)=\frac{1}{p}$.

• 推导

令$q=1-p$，则有;
$$\begin{gathered} E(X)=\sum _{x=1}^{\infty }xp{q}^{x-1} \\ =p\sum _{x=1}^{\infty }x{q}^{x-1} \\ =p\sum _{x=1}^{\infty }\frac{d(q^x)}{dq} \\ =p\frac{d}{dq}\sum _{x=0}^{\infty }{q}^x \\ =p\frac{d}{dq}(\frac{1}{1-q})=\frac{1}{p} \end{gathered}$$

方差

$Var(X)=\frac{1-p}{p^2}$

• 推导

$$\begin{gathered} E(X^2)=\sum _{x=1}^{\infty }{x}^{2}p{q}^{x-1} \\ =p\sum _{x=1}^{\infty }{x}^2{q}^{x-1} \\ =p\sum _{x=1}^{\infty }x\frac{d(q^x)}{dq} \\ =p\frac{d}{dq}\sum _{x=0}^{\infty }x{q}^x \end{gathered}$$

由无穷级数的理论可知${\sum }_{x=0}^{\infty }x{q}^x=\frac{q}{(1-q{)}^2}$

从而：

接上式:$E(X^2)=p\frac{d}{dq}(\frac{q}{(1-q{)}^2)}=\frac{2p-p^2}{p^3}=\frac{2-p}{p^2}$.从而得到方差。

二项分布——b(n,p)

$$p(x)=P(X=x)=\left(\begin{array}{c}n\\ x\end{array}\right)p^x(1-p{)}^{n-x}x=0,1,....,n$$

n=1,时的二项分布b(1,p),又称为两点分布或0-1分布。

期望

$E(X)=np$

• 推导

$$\begin{gathered} E(X)=\sum _{x=0}^{n}x\left(\begin{array}{c}n\\ x\end{array}\right)p^x(1-p{)}^{n-x} \\ =np\sum _{x=0}^n\left(\begin{array}{c}n-1\\ x-1\end{array}\right)p^{x-1}(1-p{)}^{n-x} \\ =np\sum _{x=0}^n\left(\begin{array}{c}n-1\\ x\end{array}\right)p^x(1-p{)}^{n-1-x} \\ =np[p+(1-p){]}^{n-1}=np \end{gathered}$$

方差

$Var(X)=np(1-p)$

• 推导

$$\begin{gathered} E(X^2)=\sum _{x=0}^n{x}^2\left(\begin{array}{c}n\\ x\end{array}\right)p^x(1-p{)}^{n-x} \\ =\sum _{x=2}^{n}x(x-1)\left(\begin{array}{c}n\\ x\end{array}\right)p^x(1-p{)}^{n-x}+\sum _{x=1}^{n}x\left(\begin{array}{c}n\\ x\end{array}\right)p^x(1-p{)}^{n-x} \\ =n(n-1)p^2\sum _{x=2}^n\left(\begin{array}{c}n-2\\ x-2\end{array}\right)p^{x-2}(1-p{)}^{n-x}+np \\ =n(n-1)p^2+np=n^2{p}^2+np(1-p) \end{gathered}$$

故:
$$Var(X)=E(X^2)-E(X{)}^2=np(1-p)$$

泊松分布—— $P(\lambda )$

$$P(X=x)=\frac{{\lambda }^x}{x!}{e}^{-\lambda }x=0,1,....\lambda >0$$

期望

$E(X)=\lambda$

• 推导

$$E(X)=\sum _{x=0}^{\infty }x\cdot \frac{{\lambda }^x}{x!}{e}^{-\lambda }=\lambda e^{-\lambda }\sum _{x=1}^{\infty }\frac{{\lambda }^{x-1}}{(x-1)!}=\lambda$$

方差

$Var(X)=\lambda$

• 推导

$$E(X-\lambda {)}^2=E[X^2-2\lambda X+{\lambda }^2]=E(X^2)-2\lambda E(X)+{\lambda }^2$$

又
$$\begin{gathered} E(X^2)=\sum _{x=0}^{\infty }{x}^2\cdot \frac{{\lambda }^x}{x!}{e}^{-\lambda } \\ =\sum _{x=1}^{\infty }x\cdot \frac{{\lambda }^x}{(x-1)!}{e}^{-\lambda } \\ =\sum _{x=1}^{\infty }[(x-1)+1]\cdot \frac{{\lambda }^x}{(x-1)!}{e}^{-\lambda } \\ ={\lambda }^2{e}^{-\lambda }\sum _{x=2}^{\infty }\frac{{\lambda }^{x-2}}{(x-2)!}+\lambda e^{-\lambda }\sum _{x=1}^{\infty }\frac{{\lambda }^{x-1}}{(x-1)!} \\ ={\lambda }^2+\lambda \end{gathered}$$
故：$E(X-\lambda {)}^2={\lambda }^2+\lambda -2{\lambda }^2+{\lambda }^2=\lambda$

超几何分布——h(n,N,M)

$$P(X=x)=\frac{\left(\begin{array}{c}M\\ x\end{array}\right)\left(\begin{array}{c}N-M\\ n-x\end{array}\right)}{\left(\begin{array}{c}N\\ n\end{array}\right)}x=0,1,\dots ,r,r=min(n,M)$$

期望

$E(X)=\frac{nM}{N}$

• 推导

$$\begin{array}{rl}E(X)& =\sum _{x=0}^{r}x\frac{\left(\begin{array}{c}M\\ x\end{array}\right)\left(\begin{array}{c}N-M\\ n-x\end{array}\right)}{\left(\begin{array}{c}N\\ n\end{array}\right)}\\ & =\frac{nM}{N}\sum _{x=1}^r\frac{\left(\begin{array}{c}M-1\\ x-1\end{array}\right)\left(\begin{array}{c}N-M\\ n-x\end{array}\right)}{\left(\begin{array}{c}N-1\\ n-1\end{array}\right)}\\ & =\frac{nM}{N}\end{array}$$

方差

$Var(X)=\frac{nM}{N}(1-\frac{M}{N})(\frac{N-n}{N-1})$

• 推导

$$\begin{array}{rl}E(X^2)& =\sum _{x=0}^r{x}^2\frac{\left(\begin{array}{c}M\\ x\end{array}\right)\left(\begin{array}{c}N-M\\ n-x\end{array}\right)}{\left(\begin{array}{c}N\\ n\end{array}\right)}\\ & =\frac{nM}{N}\sum _{x=1}^{r}x\frac{\left(\begin{array}{c}M-1\\ x-1\end{array}\right)\left(\begin{array}{c}N-M\\ n-x\end{array}\right)}{\left(\begin{array}{c}N-1\\ n-1\end{array}\right)}\\ & =\frac{nM}{N}\sum _{x=1}^r(x-1)\frac{\left(\begin{array}{c}M-1\\ x-1\end{array}\right)\left(\begin{array}{c}N-M\\ n-x\end{array}\right)}{\left(\begin{array}{c}N-1\\ n-1\end{array}\right)}+\frac{nM}{N}\sum _{x=1}^r\frac{\left(\begin{array}{c}M-1\\ x-1\end{array}\right)\left(\begin{array}{c}N-M\\ n-x\end{array}\right)}{\left(\begin{array}{c}N-1\\ n-1\end{array}\right)}\\ & =\frac{nM}{N}(M-1)\sum _{x=1}^r\frac{\left(\begin{array}{c}M-2\\ x-2\end{array}\right)\left(\begin{array}{c}N-M\\ n-x\end{array}\right)}{\left(\begin{array}{c}N-1\\ n-1\end{array}\right)}+\frac{nM}{N}\\ & =\frac{nM}{N}[\frac{(M-1)(n-1)}{N-1}+1]\end{array}$$

再由方差公式即可求得。

连续型随机变量

def

$p(x)$是实数轴上的一个函数，满足：

（1）$p(x)\ge 0$.（非负）

（2）${\int }_{-\infty }^{\infty }=1$

则称$p(x)$为概率密度函数。

期望：$E(X)={\int }_{-\infty }^{\infty }xp(x)dx$.

分布函数：

$F(x)=P(X\le x)={\int }_{-\infty }^{x}p(x)dx$.

分布函数的性质

1. $F(x)$是直线上的连续函数

2. $P(X=x)=0$

3. $P(a\le X\le b)=P(a\le X<b)=P(a<X\le b)=P(a<X<b)$

4. 在$F(x)$导数存在的点x 上有：

   ​ $F^{\prime }(x)=p(x)$

常见分布

均匀分布——U(a,b)

密度函数

$$p(x)=\{\begin{array}{ll}\frac{1}{b-a}& a\le x\le b\\ 0& 其他\end{array}$$

分布函数

$$F(x)=\{\begin{array}{ll}0& x<a\\ \frac{x-a}{b-a}& a\le x\le b\\ 1& x>b\end{array}$$

期望

$$E(X)={\int }_{-\infty }^{\infty }xp(x)dx={\int }_a^{b}x\cdot \frac{1}{b-a}dx=\frac{1}{b-a}\frac{x^2}{2}{|}_a^b=\frac{b^2-a^2}{2(b-a)}=\frac{a+b}{2}$$

方差

$$\begin{gathered} E(X^2)={\int }_a^b\frac{x^2}{b-a}dx \\ =\frac{1}{b-a}\frac{x^3}{3}{|}_a^b \\ =\frac{1}{3}(b^2+ab+a^2) \end{gathered}$$

故:
$$Var(X)=\frac{(b-a{)}^2}{12}$$

指数分布—— $Exp(\lambda )$

密度函数

$$p(x)=\{\begin{array}{ll}\lambda e^{-\lambda x}& x\ge 0\\ 0& x<0\end{array}$$

分布函数

$$F(x)=\{\begin{array}{ll}1-e^{-\lambda x}& x\ge 0\\ 0& x<0\end{array}$$

期望

$$E(X)={\int }_{-\infty }^{\infty }xp(x)dx={\int }_0^{\infty }\lambda x{e}^{-\lambda x}dx=\frac{1}{\lambda }$$

方差

$$Var(X)=\frac{1}{{\lambda }^2}$$

推导 见伽马分布。

柯西分布

密度函数

$$p(x)=\frac{1}{\pi (1+x^2)}$$

数学期望不存在

正态分布——$N(\mu ,{\sigma }^2)$

密度函数

$$p(x)=\frac{1}{\sqrt{2\pi }\sigma }{e}^{-\frac{(x-\mu {)}^2}{2{\sigma }^2}}-\infty <x<\infty$$

分布函数

$$F(x)=\frac{1}{\sqrt{2\pi }\sigma }{\int }_{-\infty }^x{e}^{-\frac{(x-\mu {)}^2}{2{\sigma }^2}}dx-\infty <x<\infty$$

期望

$E(X)=\mu$.

• 推导

$$\begin{gathered} 令z=\frac{x-\mu }{\sigma }, \\ E(X)=\frac{1}{\sqrt{2\pi }\sigma }{\int }_{-\infty }^{\infty }x{e}^{-\frac{(x-\mu {)}^2}{2{\sigma }^2}}dx \\ =\frac{1}{\sqrt{2\pi }\sigma }{\int }_{-\infty }^{\infty }(\sigma z+\mu )e^{-\frac{z^2}{2}}dz \\ =\frac{1}{\sqrt{2\pi }}[\sigma {\int }_{-\infty }^{\infty }z{e}^{-\frac{x^2}{2}}dz+\mu {\int }_{-\infty }^{\infty }{e}^{-\frac{x^2}{2}}dz] \\ =0+\mu =\mu \end{gathered}$$

方差

$$\begin{gathered} Var(X)=E(X-E(X){)}^2 \\ =E(X-\mu {)}^2 \\ =\frac{1}{\sqrt{2\pi }}\sigma {\int }_{-\infty }^{\infty }(x-\mu {)}^2{e}^{-\frac{(x-\mu {)}^2}{2{\sigma }^2}}dx \\ \stackrel{u=\frac{x-\mu }{\sigma }}{=}\frac{{\sigma }^2}{\sqrt{2\pi }}{\int }_{-\infty }^{\infty }{u}^2{e}^{-\frac{u^2}{2}}du \\ =\frac{2{\sigma }^2}{\sqrt{2\pi }}{\int }_0^{\infty }{u}^2{e}^{-\frac{u^2}{2}}du \\ \stackrel{y=\frac{u^2}{2}}{=}\frac{2{\sigma }^2}{\sqrt{2\pi }}\sqrt{2}{\int }_0^{\infty }{y}^{\frac{1}{2}}{e}^{-y}dy \\ =\frac{2{\sigma }^2}{\sqrt{2\pi }}\sqrt{2}\Gamma (\frac{3}{2})=\frac{2{\sigma }^2}{\sqrt{2\pi }}\frac{\sqrt{2\pi }}{2}={\sigma }^2 \end{gathered}$$

标准正态分布——N(0,1）

密度函数

$$\varphi (\mu )=\frac{1}{\sqrt{2\pi }}{e}^{-\frac{u^2}{2}}-\infty <u<\infty$$

分布函数

$$\Phi (\mu )=\frac{1}{\sqrt{2\pi }}{\int }_{-\infty }^u{e}^{-\frac{x^2}{2}}dx-\infty <u<\infty$$

• 对任意实数u，有：

$$\Phi (-u)=1-\Phi (u)$$

• 当$X\sim N(\mu ,{\sigma }^2)$时，$U=\frac{X-\mu }{\sigma }$是标准正态变量，即$U\sim N(0,1)$.

伽玛分布—— $Ga(\alpha ,\lambda )$

伽玛函数

含参数$\alpha$的积分;
$$\Gamma (\alpha )={\int }_0^{\infty }{x}^{\alpha -1}{e}^{-x}dx\alpha >0$$
性质：

1. $\Gamma (1)=1,\Gamma (\frac{1}{2})=\sqrt{\pi }$
2. 递推公式：$\Gamma (\alpha +1)=\alpha \Gamma (\alpha )$.特别对于n，$\Gamma (n+1)=n!$
3. ${\int }_0^{\infty }{x}^{\alpha -1}{e}^{-\lambda x}dx=\frac{\Gamma (\alpha )}{{\lambda }^{\alpha }}$

密度函数

$$p(x)=\{\begin{array}{ll}\frac{{\lambda }^{\alpha }}{\Gamma (\alpha )}{x}^{\alpha -1}{e}^{-\lambda x}& x>0\\ 0& x\le 0\end{array}$$

简记为;
$$p(x)=\frac{{\lambda }^{\alpha }}{\Gamma (\alpha )}{x}^{\alpha -1}{e}^{-\lambda x}x>0$$
$\alpha =1$的时候伽马分布就是指数分布，密度函数
$$p(x)=\lambda e^{-\lambda x}x>0$$
$\alpha =\frac{n}{2},\lambda =\frac{1}{2}$时伽马分布为卡方分布。

期望

$$E(X)=\frac{{\lambda }^{\alpha }}{\Gamma (\alpha )}{\int }_0^{\infty }{x}^{\alpha }{e}^{-\lambda x}dx=\frac{\Gamma (\alpha +1)}{\Gamma (\alpha )}\frac{1}{\lambda }=\frac{\alpha }{\lambda }$$

方差

$$\begin{gathered} E(X^2)=\frac{{\lambda }^{\alpha }}{\Gamma (\alpha )}{\int }_0^{\infty }{x}^2\cdot x^{\alpha }{e}^{-\lambda x}dx \\ =\frac{{\lambda }^{\alpha }}{\Gamma (\alpha )}\frac{\Gamma (\alpha +2)}{{\lambda }^{\alpha +2}} \\ =\frac{\alpha (\alpha +1)}{{\lambda }^2} \end{gathered}$$

自由度为n的${\chi }^2$分布——${\chi }^2(n)$

即伽玛分布$Ga(\frac{n}{2},\frac{1}{2})$,

密度函数

$$p(x)=\frac{1}{\Gamma (\frac{n}{2})2^{\frac{n}{2}}}{x}^{\frac{n}{2}-1}{e}^{-\frac{x}{2}}x>0$$

期望

$$E(X)=n$$

方差

$$Var(X)=2n$$

推导见伽马分布。

贝塔分布——Be(a,b)

贝塔函数

含参数a，b的积分：
$$\beta (a,b)={\int }_0^1{x}^{a-1}(1-x{)}^{b-1}dxa>0,b>0$$
性质：

1. $\beta (a,b)=\beta (b,a)$

2. 贝塔函数与伽玛函数的关系：
   $$\beta (a,b)=\frac{\Gamma (a)\Gamma (b)}{\Gamma (a+b)}$$

密度函数

$$p(x)=\frac{\Gamma (a+b)}{\Gamma (a)\Gamma (b)}{x}^{a-1}(1-x{)}^{b-1}0\le x\le 1$$

期望

$$\begin{gathered} E(X)=\frac{\Gamma (a+b)}{\Gamma (a)\Gamma (b)}{\int }_0^1{x}^{a-1}(1-x{)}^{b-1}dx \\ =\frac{\Gamma (a+b)}{\Gamma (a)\Gamma (b)}\cdot \frac{\Gamma (a+1)\Gamma (b)}{\Gamma (a+b+1)} \\ =\frac{a}{a+b} \end{gathered}$$

与均匀分布的关系

Be（1,1）就是[0,1] 上的均匀分布。

期望的性质

1. $E[cg(X)]=cE[g(X)]$
2. $E[g(X)\pm h(X)]=E[g(X)]\pm E[h(X)]$
3. $E(c)=c$,c为常数

方差与方差的性质

$Var(X)=E[X-E(X){]}^2$

性质：

1. $Var(c)=0$,c为常数
2. $Var(aX+b)=a^{2}Var(X)$
3. $Var(X)=E(X^2)-[E(X){]}^2$

多元情况后续更新。