书上给出的解答虽然步骤上是很清楚但是个具体的结果并不是很详细,为了加强理解,自己实现了解题过程的所有数据,详细代码如下:
初始化数据和初始的权重
import numpy as np
x = np.arange(10)
y = np.array([1] * 3 + [-1] * 3 + [1] * 3 + [-1])
w1 = np.array([.1] * 10)
寻找简单分类器的阈值
for v1 in np.arange(1.5, 10, 1):
G1 = [1 if i < v1 else -1 for i in range(10)]
err1 = ((G1 != y) * w1).sum()
print(v1, "时错误率:%.3f" % err1, end=' ')
G1 = [-1 if i < v1 else 1 for i in range(10)]
err1 =((G1 != y) * w1).sum()
print("错误率:%.4f"% err1)
结果如下:
这里有两个最优阈值,书上只给出了其中的一个,我们分别求出对应的结果
第一种情况: 阈值是2.5
v1 = 2.5
G1 = np.array([1 if i < v1 else -1 for i in range(10)])
err1 = round(((G1 != y) * w1).sum(), 4)
alpha1 = round(np.log((1-err1)/err1) / 2, 4)
print("alpha1:", alpha1)
f1 = np.sign(alpha1 * G1)
(f1 == y).all()
得到的分类器系数为0.4236,并没有完全预测正确,下面接着设计第二个分类器
w2 = (w1 * np.exp(-alpha1 * y * G1)) / (w1 * np.exp(-alpha1 * y * G1)).sum()
for v2 in np.arange(1.5, 10, 1):
G2 = np.array([1 if i < v2 else -1 for i in range(10)])
err2 =((G2 != y) * w2).sum()
print(v2, "时错误率:%.4f" % err2, end=' ')
G2 = np.array([-1 if i < v2 else 1 for i in range(10)])
err2 =((G2 != y) * w2).sum()
print("错误率:%.4f" % err2)
结果为:
阈值最优的是8.5
v2 = 8.5
G2 = np.array([1 if i < v2 else -1 for i in range(10)])
err2 = round(((G2 != y) * w2).sum(), 4)
alpha2 = round(np.log((1-err2)/err2) / 2, 4)
print("alpha2:", alpha2)
f2 = np.sign(alpha1 * G1 + alpha2 * G2)
(f2 == y).all()
第二个分类器的系数是0.6496, 仍然没有完全预测正确,需要设计第三个分类器
w3 = (w2 * np.exp(-alpha2 * y * G2)) / (w2 * np.exp(-alpha2 * y * G2)).sum()
for v3 in np.arange(1.5, 10, 1):
G3 = np.array([1 if i < v3 else -1 for i in range(10)])
err3 =((G3 != y) * w3).sum()
print(v3, "时前正后负的错误率:%.4f " % err3, end=' ')
G3 = np.array([-1 if i < v3 else 1 for i in range(10)])
err3 =((G3 != y) * w3).sum()
print("前负后正的错误率:%.4f " % err3)
结果
阈值最优的是5.5,但是注意是前负后正
v3 = 5.5
G3 = np.array([-1 if i < v3 else 1 for i in range(10)])
err3 = round(((G3 != y) * w3).sum(), 4)
alpha3 = round(np.log((1 - err3) / err3) / 2, 4)
print("apha3:", alpha3)
f3 = np.sign(alpha1 * G1 + alpha2 * G2 + alpha3 * G3)
(f3 == y).all()
结果:
第三个分类器的系数是0.7521,分类器完全预测正确.
综上, 最终的分类器是:
f = 0.4236 * G1 + 0.6496 * G2 + 0.7521 * G3
第二种情况: 阈值是8.5
v1 = 8.5
G1 = np.array([1 if i < v1 else -1 for i in range(10)])
err1 = 0.3
alpha1 = round(np.log((1-err1)/err1) / 2, 4)
print("alpha1:", alpha1) # 0.4236
f1 = np.sign(alpha1 * G1)
(f1 == y).all()
得到的分类器系数为0.4236,并没有完全预测正确,下面接着设计第二个分类器
w2 = (w1 * np.exp(-alpha1 * y * G1)) / (w1 * np.exp(-alpha1 * y * G1)).sum()
for v2 in np.arange(1.5, 10, 1):
G2 = np.array([1 if i < v2 else -1 for i in range(10)])
err2 =((G2 != y) * w2).sum()
print(v2, "时错误率:%.4f" % err2, end=' ')
G2 = np.array([-1 if i < v2 else 1 for i in range(10)])
err2 =((G2 != y) * w2).sum()
print("错误率:%.4f" % err2)
结果:
第二个分类器阈值最优的是2.5
v2 = 2.5
G2 = np.array([1 if i < v2 else -1 for i in range(10)])
err2 = 0.2142857
alpha2 = round(np.log((1-err2)/err2) / 2, 4)
# 0.6496
print("alpha2:", alpha2)
f2 = np.sign(alpha1 * G1 + alpha2 * G2)
(f2 == y).all()
第二个分类器的系数是0.6496, 仍然没有完全预测正确,需要设计第三个分类器
w3 = (w2 * np.exp(-alpha2 * y * G2)) / (w2 * np.exp(-alpha2 * y * G2)).sum()
for v3 in np.arange(1.5, 10, 1):
G3 = np.array([1 if i < v3 else -1 for i in range(10)])
err3 =((G3 != y) * w3).sum()
print(v3, "时错误率:%.4f " % err3, end=' ')
G3 = np.array([-1 if i < v3 else 1 for i in range(10)])
err3 =((G3 != y) * w3).sum()
print("错误率:%.4f " % err3)
第三个分类器阈值最优的是5.5, 但是注意是前负后正
v3 = 5.5
G3 = np.array([-1 if i < v3 else 1 for i in range(10)])
err3 = 0.18182
alpha3 = round(np.log((1-err3)/err3) / 2, 4)
print("apha3:", alpha3)
f3 = np.sign(alpha1 * G1 + alpha2 * G2 + alpha3 * G3)
(f3 == y).all()
第三个分类器的系数是0.752, 分类器完全预测正确.
综上, 最终的分类器是:
f = 0.4236 * G1 + 0.6496 * G2 + 0.7521 * G3
可见, 虽然每个分类器的阈值不一致, 但是最终的结果却是相同的,最终的分类器均是:
f = sign(0.4236 * G1 + 0.6496 * G2 + 0.7521 * G3)
