From 6b7b6ea60edb080b1d63f8c281ff4f38e13f823d Mon Sep 17 00:00:00 2001 From: zhangy Date: Thu, 5 Jun 2025 00:41:38 +0000 Subject: [PATCH] =?UTF-8?q?=E6=9B=B4=E6=96=B0=20test.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- test.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test.md b/test.md index 4101b28..413a56e 100644 --- a/test.md +++ b/test.md @@ -11,7 +11,7 @@ - 交叉熵 (cross entropy): $H(p,q)=\sum -p(x_i) \log q(x_i)$ - 相对熵 (relative entropy) 或者 KL 散度 (Kullback-Leibler divergence, KLD): $D_{KL}(p||q):=H(p,q)-H(p)= \sum -p(x_i)\log \frac{q(x_i)}{p(x_i)}$ - $g(X,Y)=D_{KL}(p(x,y)||p(x)p(y))$ -- JS 散度:$JS:=1/2 ( D_{KL}(p||m) + D_{KL}(q||m) )$, 这里$m=(p+q)/2$。 p和q支撑集不相交时,JS 散度为一个常数 log2 +- JS 散度:$JS:=1/2 ( D_{KL}(p||m) + D_{KL}(q||m) )$, 这里$m=(p+q)/2 $ 。 p和q支撑集不相交时,JS 散度为一个常数 log2 - 最小化交叉熵=最小化KL=极大似然 ### Bias-Variance tradeoff