我现在正在自学机器学习。
关于信息增益有一个简单的问题。如何从图片中的数据计算信息增益？我不明白这一点。
有谁能解释一下如何从第一行得到0.992385吗？
非常感谢！
回答：
设 g(x) = -x * log(x) /log(2)
总人数为48842。
H(wealth, relation) = g[52/48842] + g[111/48842] + g[309/48842] + g[1093/48842] + g[1238/48842] + g[1276/48842] + g[1454/48842] + g[4816/48842] + g[7470/48842] + g[8846/48842] + g[10870/48842] + g[11307/48842] = 2.7835
H(wealth) = g(poor/total) + g(rich/total) = g[0.239282] + g[0.760718] = 0.793844
H(relation) = g(Husband/total) + g(Not_in_familly/total) + … = g[0.0308341] + g[0.0477253] + g[0.10493] + g[0.155215] + g[0.257627] + g[0.403669] = 2.15508
H(wealth | relation) = H(wealth, relation) – H(relation) = 2.7835 – 2.15508 = 0.628421
IG = H(wealth) – H(wealth|relation) = H(wealth) + H(relation) – H(wealth, relation) = 0.165423
这是用Mathematica编写的源代码。如果您觉得需要看到其他语言的源代码，请在下方评论中注明您偏好的语言。如果我有时间，我会编写出来。- 祝好，Hans
Mathematica中的源代码
(* =================================================== *)
m = {{10870, 8846}, {11307, 1276}, {1454, 52}, {7470, 111}, {4816, 309}, {1238, 1093}};iTot = Total[ Flatten[m]];h[x_] := -x * Log[2, x];fHAll = Sum[ h[ m[[i, j]]/ iTot ], {i, 6}, {j, 2}] // N;fHWealth = h[ Total[ m[[All, 1]]]/iTot] + h[ Total[ m[[All, 2]]]/iTot] // N ;fHRelation = Sum[ h[ Total[ m[[i ]]]/iTot] , {i, Length[m]}] // N;fWealthGivenRelation = fHAll - fHRelation;Print[" H(relation, wealth) = ", fHAll];Print[" H(relation) = ", fHRelation];Print[" H(wealth) = ", fHWealth];Print[" H(wealth | relation) = ", fWealthGivenRelation];Print[" IG = MI = ", fHWealth - fWealthGivenRelation, " = ", fHWealth + fHRelation - fHAll];(* ===================output ====================*)
H(relation, wealth) = 2.7835 H(relation) = 2.15508 H(wealth) = 0.793844 H(wealth | relation) = 0.628421 IG = MI = 0.165423 = 0.165423哦，我没有回答你的主要问题。这里是答案。
H(wealth | relation = husband) = g(10870/(10870 + 8846)) + g(8846/(10870 + 8846)) = 0.992385
