admin/test
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

更新 机器学习.md

Browse Source
This commit is contained in:
admin 2025-06-10 06:42:17 +00:00
parent d5b3548429
commit c5e2383475
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
机器学习.md
View File

@ -11,7 +11,7 @@
- 交叉熵 (cross entropy): $H(p,q)=\sum -p(x_i) \log q(x_i)$ - 交叉熵 (cross entropy): $H(p,q)=\sum -p(x_i) \log q(x_i)$
- 相对熵 (relative entropy) 或者 KL 散度 (Kullback-Leibler divergence, KLD): $D_{KL}(p||q):=H(p,q)-H(p)= \sum -p(x_i)\log \frac{q(x_i)}{p(x_i)}$ - 相对熵 (relative entropy) 或者 KL 散度 (Kullback-Leibler divergence, KLD): $D_{KL}(p||q):=H(p,q)-H(p)= \sum -p(x_i)\log \frac{q(x_i)}{p(x_i)}$
- $g(X,Y)=D_{KL}(p(x,y)||p(x)p(y))$ - $g(X,Y)=D_{KL}(p(x,y)||p(x)p(y))$
- JS 散度：$JS:=1/2 ( D_{KL}(p||m) + D_{KL}(q||m) )$, 这里$m=(p+q)/2$ p和q支撑集不相交时，JS 散度为一个常数 log2 - JS 散度：$JS:=1/2 ( D_{KL}(p||m) + D_{KL}(q||m) )$, 这里$m=(p+q)/2$ 。p和q支撑集不相交时，JS 散度为一个常数 log2
- 最小化交叉熵=最小化KL=极大似然 - 最小化交叉熵=最小化KL=极大似然
### Bias-Variance tradeoff ### Bias-Variance tradeoff