网络程序设计学习心得
项目跟进情况
A1：神经网络实现手写字符识别系统
这个项目虽然没有检查，但我还是认真地阅读了源代码，对神经网络图像识别有了大概的了解。通俗地说，这个程序就是把一幅图像按照像素点变换为一个向量，之后进行拟合。
我尝试过对它进行了一些小的改进，比如用ReLU函数替换sigmoid函数，即把ocr.py改为：
# -*- coding: UTF-8 -*-
import csv
import numpy as np
from numpy import matrix
from math import pow
from collections import namedtuple
import math
import random
import os
import json
class OCRNeuralNetwork:
LEARNING_RATE = 0.1
WIDTH_IN_PIXELS = 20
# 保存神经网络的文件路径
NN_FILE_PATH = 'nn.json'
def __init__(self, num_hidden_nodes, data_matrix, data_labels, training_indices, use_file=True):
# relu函数
self.relu = np.vectorize(self._relu_scalar)
# relu求导函数
self.relu_prime = np.vectorize(self._relu_prime_scalar)
# 决定了要不要导入nn.json
self._use_file = use_file
# 数据集
self.data_matrix = data_matrix
self.data_labels = data_labels
if (not os.path.isfile(OCRNeuralNetwork.NN_FILE_PATH) or not use_file):
# 初始化神经网络
self.theta1 = self._rand_initialize_weights(400, num_hidden_nodes)
self.theta2 = self._rand_initialize_weights(num_hidden_nodes, 10)
self.input_layer_bias = self._rand_initialize_weights(1, num_hidden_nodes)
self.hidden_layer_bias = self._rand_initialize_weights(1, 10)
# 训练并保存
TrainData = namedtuple('TrainData', ['y0', 'label'])
self.train([TrainData(self.data_matrix[i], int(self.data_labels[i])) for i in training_indices])
self.save()
else:
# 如果nn.json存在则加载
self._load()
def _rand_initialize_weights(self, size_in, size_out):
return [((x * 0.12) - 0.06) for x in np.random.rand(size_out, size_in)]
def _relu_scalar(self, z):
if(z>0):
return z
return 0
def _relu_prime_scalar(self, z):
if(z>0):
return 1
return 0
def train(self, training_data_array):
for data in training_data_array:
# 前向传播得到结果向量
y1 = np.dot(np.mat(self.theta1), np.mat(data.y0).T)
sum1 = y1 + np.mat(self.input_layer_bias)
y1 = self.relu(sum1)
y2 = np.dot(np.array(self.theta2), y1)
y2 = np.add(y2, self.hidden_layer_bias)
y2 = self.relu(y2)
# 后向传播得到误差向量
actual_vals = [0] * 10
actual_vals[data.label] = 1
output_errors = np.mat(actual_vals).T - np.mat(y2)
hidden_errors = np.multiply(np.dot(np.mat(self.theta2).T, output_errors), self.relu_prime(sum1))
# 更新权重矩阵与偏置向量
self.theta1 += self.LEARNING_RATE * np.dot(np.mat(hidden_errors), np.mat(data.y0))
self.theta2 += self.LEARNING_RATE * np.dot(np.mat(output_errors), np.mat(y1).T)
self.hidden_layer_bias += self.LEARNING_RATE * output_errors
self.input_layer_bias += self.LEARNING_RATE * hidden_errors
def predict(self, test):
y1 = np.dot(np.mat(self.theta1), np.mat(test).T)
y1 = y1 + np.mat(self.input_layer_bias) # Add the bias
y1 = self.relu(y1)
y2 = np.dot(np.array(self.theta2), y1)
y2 = np.add(y2, self.hidden_layer_bias) # Add the bias
y2 = self.relu(y2)
results = y2.T.tolist()[0]
return results.index(max(results))
def save(self):
if not self._use_file:
return
json_neural_network = {
"theta1":[np_mat.tolist()[0] for np_mat in self.theta1],
"theta2":[np_mat.tolist()[0] for np_mat in self.theta2],
"b1":self.input_layer_bias[0].tolist()[0],
"b2":self.hidden_layer_bias[0].tolist()[0]
};
with open(OCRNeuralNetwork.NN_FILE_PATH,'w') as nnFile:
json.dump(json_neural_network, nnFile)
def _load(self):
if not self._use_file:
return
with open(OCRNeuralNetwork.NN_FILE_PATH) as nnFile:
nn = json.load(nnFile)
self.theta1 = [np.array(li) for li in nn['theta1']]
self.theta2 = [np.array(li) for li in nn['theta2']]
self.input_layer_bias = [np.array(nn['b1'][0])]
self.hidden_layer_bias = [np.array(nn['b2'][0])]
但是很遗憾效果没有明显地提升，所以没有pr。
A2:血常规检验报告的图像OCR识别
现在采用的程序的基本思想是调用CV2模块的findContours提取矩形轮廓，筛选对角线大于阈值的轮廓，而我计划采用霍夫变换进行图像检测，但只写了一部分，最后完成的代码为(Python)：
# -*- coding:utf-8 -*-
import cv2
import numpy as np
import traceback
def houghtransform(path):
'''使用霍夫变换来检测直线，path为图片存储路径'''
img = cv2.imread(path)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
#print('成功转换为灰度图...')
edges = cv2.Canny(gray, 100, 175, apertureSize=3)
'''边缘检测'''
midpointlist = []
lines = cv2.HoughLines(edges, 1, np.pi / 180, 200)
for rho, theta in lines[0]:
a = np.cos(theta)
b = np.sin(theta)
x0 = a * rho
y0 = b * rho
x1 = float(x0 + 5000 * (-b))
y1 = float(y0 + 5000 * (a))
x2 = float(x0 - 5000 * (-b))
y2 = float(y0 - 5000 * (a))
#cv2.line(img,(x1,y1),(x2,y2),(0,0,255),2)
if(x2-x1==0):
continue
k = (y2 - y1) / (x2 - x1)
if(k>0.5 or k<-0.5):
'''这里本来是要求用户拍照保持三条主要直线水平，否则不予通过。当然现在采用的程序可以对图像进行旋转操作'''
continue
b = y1 - k * x1
midpoint = k * (img.shape[1] / 2) + b
midpointlist.append(midpoint)
return midpointlist
def midpointcluster(pointlist):
'''把直线聚簇为三条主要直线'''
clist = sorted(pointlist)
#print clist
clist2 = []
clist3 = []
gap = 30
for i in range(len(clist) - 1):
clist2.append(clist[i + 1] - clist[i])
for i in range(len(clist2)):
if (clist2[i] > gap):
clist3.append(i)
#print clist3
if (len(clist3) != 2):
raise Exception('Number of line Error')
midpoint1 = 0
midpoint2 = 0
midpoint3 = 0
for i in range(len(clist)):
if (i <= clist3[0]):
midpoint1 = midpoint1 + clist[i]
elif (i <= clist3[1]):
midpoint2 = midpoint2 + clist[i]
else:
midpoint3 = midpoint3 + clist[i]
midpointcluster=(midpoint1,midpoint2,midpoint3)
if(midpoint3-midpoint2<midpoint2-midpoint1):
raise Exception('Distance between line Error')
return midpointcluster
效果如下
本来打算做一个OCR项目，但无奈最后虎头蛇尾。究其原因，一方面是自己编程能力还有待提高，另一方面，是自己高级软件工程没学好，对这个程序没有一个整体的规划，只是随便看了一些将图像处理的书，看了“霍夫变换”觉得可以拿来用，就匆匆写程序，而对于程序没有一个整体的规划，搞得自己最后都不知道自己写的程序是要干什么。还有就是要及时把自己的项目库更新，不要闭门造车。
虽然程序没有写出来，但是自己还是收获了一些图像处理方面的知识。例如霍夫变换(Hough Transform)是图像处理中从图像中识别几何形状的基本方法之一，应用很广泛，也有很多改进算法。主要用来从图像中分离出具有某种相同特征的几何形状（如直线，圆等）。霍夫变换在检验已知形状的目标方面具有受曲线间断影响小和不受图形旋转的影响的优点，即使目标有稍许缺损或污染也能被正确识别。
程序里用到的cv2是OpenCV提供给Python的接口，实现了图像处理和计算机视觉方面的很多通用算法。
A3：根据血常规检验的各项数据预测年龄和性别
这里我用了三种方法来预测性别：
1.matlab神经网络
（已被接受）
因为对Python实在是不太熟悉，这个项目里我利用matlab的神经网络工具箱搭建了一个简单的神经网络，准确率69.5%
在matlab里面利用newff函数即可建立一个前馈（Feed-forward）神经网络，它的语法如下：
Syntax
net = newff(PR,[S1 S2...SNl],{TF1 TF2...TFNl},BTF,BLF,PF)
Description
NEWFF(PR,[S1 S2...SNl],{TF1 TF2...TFNl},BTF,BLF,PF) takes,
PR - Rx2 matrix of min and max values for R input elements.
Si - Size of ith layer, for Nl layers.
TFi - Transfer function of ith layer, default = 'tansig'.
BTF - Backprop network training function, default = 'trainlm'.
BLF - Backprop weight/bias learning function, default = 'learngdm'.
PF - Performance function, default = 'mse'.
and returns an N layer feed-forward backprop network.
训练网络需要用到train函数，它的用法为：
train
trains a network net according to net.trainFcn and net.trainParam.
[net,tr] = train(net,X,T,Xi,Ai,EW) takes
net Network
X Network inputs
T Network targets (default = zeros)
Xi Initial input delay conditions (default = zeros)
Ai Initial layer delay conditions (default = zeros)
EW Error weights
and returns
net Newly trained network
tr Training record (epoch and perf)
这个网络示意如下：
网络的相关参数为
Network Type:Feed-forward backprop
（BP神经网络）
Training function:TRAINLM
（matlab使用较多的training function有①traingdm:带动量的梯度下降法②trainlm：Levenberg-Marquardt反向传播算法③trainscg：量化共轭梯度法）
Adaption learning function:LEARNGDM
（动量梯度下降权值和阈值学习函数）
Performance function:MSE
（Minimum Squared-Error，最小平方误差）
Transfer Function:TANSIG
（传递函数为tansig(n) = 2/(1+exp(-2*n))-1，图像如下
)
训练参数：
Performance：
训练结果
w1
[0.22934 0.33237 0.18616 -0.16863 -0.74814 -0.028468 -0.70639 -0.11416 0.1239 0.072537 0.42827 -0.45381 -0.466 0.36981 -0.22276 0.393 0.4441 -0.34499 0.47837 -0.27545 -0.021114 0.12907 0.11067 0.23152 0.22918 -0.38689 0.005718;
-0.27896 0.086497 -0.4368 -0.26154 -0.65659 -0.2295 -0.42455 0.25364 0.26732 0.41054 0.2491 0.003718 0.29398 -0.59387 0.053115 -0.2438 0.15349 0.27665 -0.21539 -0.51924 -0.026642 0.54431 -0.48744 -0.19116 0.43857 0.062959 0.46212; -0.21614 -0.091768 -0.037967 0.093128 -0.34719 0.40308 -0.32051 -0.47913 -0.67289 0.19187 0.3488 -0.45831 0.6334 -0.21682 0.42005 0.65177 -0.018414 0.29537 0.26432 0.24032 -0.40605 0.45256 -0.15487 -0.23032 0.0018145 0.6559 0.52; 0.070064 -0.36313 -0.45388 -0.24794 0.11777 -0.20669 -0.19104 -0.68718 -0.55936 0.0013916 0.55156 0.10725 0.086604 0.13864 -0.56944 -0.002838 -0.44712 -0.18217 -0.34849 -0.43119 0.22334 0.63498 -0.36183 0.34113 -0.13031 -0.65504 0.49411;
1.3085 -0.26792 -0.49039 -0.10928 0.6573 0.578 -0.55018 -0.89083 -0.42546 0.78246 -0.39543 0.033607 -1.4683 -0.351 -0.45626 -0.49475 -0.55446 0.26089 -0.047364 -0.019495 -1.5678 -0.22787 0.76314 0.49357 -0.22347 0.68213 0.23232;
0.25732 -0.032014 0.22697 -0.38135 0.016032 0.43465 -0.021983 0.39153 0.092924 -0.32182 -0.12286 -0.22144 0.35455 0.0059394 -0.56576 0.3121 0.47204 -0.26088 0.43553 -0.087725 -0.047741 -0.56659 -0.60782 0.069005 0.51185 0.044621 -0.34378;
-0.061419 0.1484 -0.044321 -0.12147 -0.53218 0.10166 -0.62919 0.32173 0.8182 -0.24366 0.73224 0.083956 -0.69837 -0.070769 0.21664 -0.93088 0.04608 0.63501 0.082489 -0.69209 -0.47494 -0.93254 -0.36528 0.41388 -0.3923 0.34204 0.61286; -1.2659 0.04718 -0.037345 0.38109 -0.069197 0.80741 -0.45849 -0.56029 -0.024898 -0.60605 -0.36698 -0.086208 -0.7393 0.068079 -0.44124 0.69219 0.14039 0.1328 -0.2807 0.42572 -1.6392 -0.60991 -2.1485 -0.24379 -0.21266 -0.26429 0.12916; -0.60882 -0.3124 0.33258 0.25615 0.25172 0.60335 0.43706 0.12853 -0.14065 0.4002 0.47063 -0.23728 -0.31461 -0.37416 -0.22733 0.25784 -0.28286 -0.43686 0.25516 0.38196 -0.49478 -0.51633 0.090486 -0.029599 -0.021952 0.074578 0.28458; -0.38579 0.3544 0.28693 0.4854 0.16293 0.30611 -0.24616 -0.10266 0.24895 -0.34168 -0.015138 -0.1131 -0.37139 0.25221 -0.00058912 -0.051016 0.3608 0.25396 -0.15008 -0.39694 0.26855 -0.0022477 0.02871 0.50095 0.024807 -0.44216 0.49674]
w2
[-0.68377 -0.13765 -0.59508 1.267 -1.8024 -0.82091 1.7094 -1.6417 -0.39652 -0.024325]
b1
[1.518; 1.2757; 0.79315; -0.41168; 0.78636; -0.2075; -0.3619; 1.1626; -1.3725; -1.4656]
b2
[0.85601]
测试程序为
load('predict_input_transpose.mat')
load('predict_output_transpose.mat')
load('train_input_transpose.mat')
load('train_output_transpose.mat')
load('network_hit139.mat')
predict_result = sim( network_hit139, predict_input_transpose);
%对网络进行测试
avg=mean(predict_result(:));
for i=1:200
if predict_result(1,i)>avg
predict_result(1,i)=1;
else predict_result(1,i)=-1;
end
end
%输出为[-1,1]之间连续的值，所以还要进行二值化
accuracy=0;
for i=1:200
if predict_result(1,i)==predict_output_transpose(1,i)
accuracy=accuracy+1;
end
end
accuracy/200
%测试准确率虽然matlab没有开源，但是只要网络的各项参数决定了，根据公式
Z1= w1*x +b1
Z2=w2*Z1+b2
…
可以其他编程语言把网络模型复制。
2 采用SMO算法的SVM
从SVM提出到现在，出现了很多优化训练的方法。其中，非常出名的一个是1982年由Microsoft Research的John C. Platt在论文《Sequential Minimal Optimization: A Fast Algorithm for TrainingSupport Vector Machines》中提出的Sequential Minimal Optimization序列最小化优化算法，简称SMO算法。我根据资料自己写了一个预测年龄的算法，但效果不太理想，准确率很低(主要是收敛程度不够)。虽然这个程序比较失败，但并非一无是处，查阅资料后，它启发我可以尝试利用pca降维提升准确度。
这个程序代码如下：
SVM.py
# -*- coding: UTF-8 -*-
from numpy import *
import time
import matplotlib.pyplot as plt
# calulate kernel value
def calcKernelValue(matrix_x, sample_x, kernelOption):
kernelType = kernelOption[0]
numSamples = matrix_x.shape[0]
kernelValue = mat(zeros((numSamples, 1)))
if kernelType == 'linear':
kernelValue = matrix_x * sample_x.T
elif kernelType == 'rbf':
sigma = kernelOption[1]
if sigma == 0:
sigma = 1.0
for i in xrange(numSamples):
diff = matrix_x[i, :] - sample_x
kernelValue[i] = exp(diff * diff.T / (-2.0 * sigma**2))
else:
raise NameError('Not support kernel type! You can use linear or rbf!')
return kernelValue
# calculate kernel matrix given train set and kernel type
def calcKernelMatrix(train_x, kernelOption):
numSamples = train_x.shape[0]
kernelMatrix = mat(zeros((numSamples, numSamples)))
for i in xrange(numSamples):
kernelMatrix[:, i] = calcKernelValue(train_x, train_x[i, :], kernelOption)
return kernelMatrix
# define a struct just for storing variables and data
class SVMStruct:
def __init__(self, dataSet, labels, C, toler, kernelOption):
self.train_x = dataSet # each row stands for a sample
self.train_y = labels # corresponding label
self.C = C # slack variable
self.toler = toler # termination condition for iteration
self.numSamples = dataSet.shape[0] # number of samples
self.alphas = mat(zeros((self.numSamples, 1))) # Lagrange factors for all samples
self.b = 0
self.errorCache = mat(zeros((self.numSamples, 2)))
self.kernelOpt = kernelOption
self.kernelMat = calcKernelMatrix(self.train_x, self.kernelOpt)
# calculate the error for alpha k
def calcError(svm, alpha_k):
output_k = float(multiply(svm.alphas, svm.train_y).T * svm.kernelMat[:, alpha_k] + svm.b)
error_k = output_k - float(svm.train_y[alpha_k])
return error_k
# update the error cache for alpha k after optimize alpha k
def updateError(svm, alpha_k):
error = calcError(svm, alpha_k)
svm.errorCache[alpha_k] = [1, error]
# select alpha j which has the biggest step
def selectAlpha_j(svm, alpha_i, error_i):
svm.errorCache[alpha_i] = [1, error_i] # mark as valid(has been optimized)
candidateAlphaList = nonzero(svm.errorCache[:, 0].A)[0] # mat.A return array
maxStep = 0; alpha_j = 0; error_j = 0
# find the alpha with max iterative step
if len(candidateAlphaList) > 1:
for alpha_k in candidateAlphaList:
if alpha_k == alpha_i:
continue
error_k = calcError(svm, alpha_k)
if abs(error_k - error_i) > maxStep:
maxStep = abs(error_k - error_i)
alpha_j = alpha_k
error_j = error_k
# if came in this loop first time, we select alpha j randomly
else:
alpha_j = alpha_i
while alpha_j == alpha_i:
alpha_j = int(random.uniform(0, svm.numSamples))
error_j = calcError(svm, alpha_j)
return alpha_j, error_j
# the inner loop for optimizing alpha i and alpha j
def innerLoop(svm, alpha_i):
error_i = calcError(svm, alpha_i)
### check and pick up the alpha who violates the KKT condition
## satisfy KKT condition
# 1) yi*f(i) >= 1 and alpha == 0 (outside the boundary)
# 2) yi*f(i) == 1 and 0<alpha< C (on the boundary)
# 3) yi*f(i) <= 1 and alpha == C (between the boundary)
## violate KKT condition
# because y[i]*E_i = y[i]*f(i) - y[i]^2 = y[i]*f(i) - 1, so
# 1) if y[i]*E_i < 0, so yi*f(i) < 1, if alpha < C, violate!(alpha = C will be correct)
# 2) if y[i]*E_i > 0, so yi*f(i) > 1, if alpha > 0, violate!(alpha = 0 will be correct)
# 3) if y[i]*E_i = 0, so yi*f(i) = 1, it is on the boundary, needless optimized
if (svm.train_y[alpha_i] * error_i < -svm.toler) and (svm.alphas[alpha_i] < svm.C) or (svm.train_y[alpha_i] * error_i > svm.toler) and (svm.alphas[alpha_i] > 0):
# step 1: select alpha j
alpha_j, error_j = selectAlpha_j(svm, alpha_i, error_i)
alpha_i_old = svm.alphas[alpha_i].copy()
alpha_j_old = svm.alphas[alpha_j].copy()
# step 2: calculate the boundary L and H for alpha j
if svm.train_y[alpha_i] != svm.train_y[alpha_j]:
L = max(0, svm.alphas[alpha_j] - svm.alphas[alpha_i])
H = min(svm.C, svm.C + svm.alphas[alpha_j] - svm.alphas[alpha_i])
else:
L = max(0, svm.alphas[alpha_j] + svm.alphas[alpha_i] - svm.C)
H = min(svm.C, svm.alphas[alpha_j] + svm.alphas[alpha_i])
if L == H:
return 0
# step 3: calculate eta (the similarity of sample i and j)
eta = 2.0 * svm.kernelMat[alpha_i, alpha_j] - svm.kernelMat[alpha_i, alpha_i] - svm.kernelMat[alpha_j, alpha_j]
if eta >= 0:
return 0
# step 4: update alpha j
svm.alphas[alpha_j] -= svm.train_y[alpha_j] * (error_i - error_j) / eta
# step 5: clip alpha j
if svm.alphas[alpha_j] > H:
svm.alphas[alpha_j] = H
if svm.alphas[alpha_j] < L:
svm.alphas[alpha_j] = L
# step 6: if alpha j not moving enough, just return
if abs(alpha_j_old - svm.alphas[alpha_j]) < 0.00001:
updateError(svm, alpha_j)
return 0
# step 7: update alpha i after optimizing aipha j
svm.alphas[alpha_i] += svm.train_y[alpha_i] * svm.train_y[alpha_j]* (alpha_j_old - svm.alphas[alpha_j])
# step 8: update threshold b
b1 = svm.b - error_i - svm.train_y[alpha_i] * (svm.alphas[alpha_i] - alpha_i_old)* svm.kernelMat[alpha_i, alpha_i]- svm.train_y[alpha_j] * (svm.alphas[alpha_j] - alpha_j_old)* svm.kernelMat[alpha_i, alpha_j]
b2 = svm.b - error_j - svm.train_y[alpha_i] * (svm.alphas[alpha_i] - alpha_i_old)* svm.kernelMat[alpha_i, alpha_j] - svm.train_y[alpha_j] * (svm.alphas[alpha_j] - alpha_j_old)* svm.kernelMat[alpha_j, alpha_j]
if (0 < svm.alphas[alpha_i]) and (svm.alphas[alpha_i] < svm.C):
svm.b = b1
elif (0 < svm.alphas[alpha_j]) and (svm.alphas[alpha_j] < svm.C):
svm.b = b2
else:
svm.b = (b1 + b2) / 2.0
# step 9: update error cache for alpha i, j after optimize alpha i, j and b
updateError(svm, alpha_j)
updateError(svm, alpha_i)
return 1
else:
return 0
# the main training procedure
def trainSVM(train_x, train_y, C, toler, maxIter, kernelOption = ('rbf', 1.0)):
# calculate training time
startTime = time.time()
# init data struct for svm
svm = SVMStruct(mat(train_x), mat(train_y), C, toler, kernelOption)
# start training
entireSet = True
alphaPairsChanged = 0
iterCount = 0
# Iteration termination condition:
# Condition 1: reach max iteration
# Condition 2: no alpha changed after going through all samples,
# in other words, all alpha (samples) fit KKT condition
while (iterCount < maxIter) and ((alphaPairsChanged > 0) or entireSet):
alphaPairsChanged = 0
# update alphas over all training examples
if entireSet:
for i in xrange(svm.numSamples):
alphaPairsChanged += innerLoop(svm, i)
print '---iter:%d entire set, alpha pairs changed:%d' % (iterCount, alphaPairsChanged)
iterCount += 1
# update alphas over examples where alpha is not 0 & not C (not on boundary)
else:
nonBoundAlphasList = nonzero((svm.alphas.A > 0) * (svm.alphas.A < svm.C))[0]
for i in nonBoundAlphasList:
alphaPairsChanged += innerLoop(svm, i)
print '---iter:%d non boundary, alpha pairs changed:%d' % (iterCount, alphaPairsChanged)
iterCount += 1
# alternate loop over all examples and non-boundary examples
if entireSet:
entireSet = False
elif alphaPairsChanged == 0 :
entireSet = True
print 'Congratulations, training complete! Took %fs!' % (time.time() - startTime)
return svm
# testing your trained svm model given test set
def testSVM(svm, test_x, test_y):
test_x = mat(test_x)
test_y = mat(test_y)
#print test_x.shape
numTestSamples = test_y.shape[0]
supportVectorsIndex = nonzero(svm.alphas.A > 0)[0]
supportVectors = svm.train_x[supportVectorsIndex]
supportVectorLabels = svm.train_y[supportVectorsIndex]
supportVectorAlphas = svm.alphas[supportVectorsIndex]
matchCount = 0
for i in xrange(numTestSamples):
kernelValue = calcKernelValue(supportVectors, test_x[i, :], svm.kernelOpt)
predict = kernelValue.T * multiply(supportVectorLabels, supportVectorAlphas) + svm.b
if sign(predict) == sign(test_y[i]):
matchCount += 1
accuracy = float(matchCount) / numTestSamples
return accuracy test_SVM.py
# -*- coding: UTF-8 -*-
from numpy import *
import sys
sys.path.append('/home/crd/桌面/1')
import SVM
################## test svm #####################
## step 1: load data
print "step 1: load data..."
dataSet = []
labels = []
fileIn = open('/home/crd/桌面/1/data1.txt')
for line in fileIn.readlines():
lineArr = line.strip( ).split('\t')
for i in range(27):
#print(lineArr[i])
dataSet.append([float(lineArr[i])])
labels.append(float(lineArr[27]))
dataSet = mat(dataSet)
labels = mat(labels).T
train_x = dataSet[0:1859, :]
train_y = labels[0:1859, :]
test_x = dataSet[1858:2059, :]
test_y = labels[1858:2059, :]
''' print test_x.shape print test_y.shape '''
## step 2: training...
print "step 2: training..."
C = 0.6
toler = 0.001
maxIter = 100
svmClassifier = SVM.trainSVM(train_x, train_y, C, toler, maxIter, kernelOption = ('linear', 0))
## step 3: testing
print "step 3: testing..."
accuracy = SVM.testSVM(svmClassifier, test_x, test_y)
## step 4: show the result
print "step 4: show the result..."
print 'The classify accuracy is: %.3f%%' % (accuracy * 100) 3 利用pca降维的SVM
（已被接受）
PCA（Principal Component Analysis）是一种常用的数据分析方法。PCA通过线性变换将原始数据变换为一组各维度线性无关的表示，可用于提取数据的主要特征分量，常用于高维数据的降维。
matlab代码如下：
load('predict_input.mat');
load('predict_output.mat');
load('train_input.mat');
load('train_output.mat');
data=[train_input;predict_input];
%因为需要对数据进行一些处理，我们把训练集和预测集合并
sex=[train_output;predict_output];
checkpoint=1600;
%利用前1600个数据训练，后面的数据进行预测
std_data=zscore(data);
%标准化
[coeff,score,latent,tsguared,explained]=pca(data);
%pca降维
re_train=score(:,1:10);
%选取影响最大的10个分量，因为它们pca累计方差95%
trunc1_train=re_train(1:checkpoint,:);
%利用checkpoint前的数据训练
trunc1_sex=sex(1:checkpoint,:);
trunc2_train=re_train(checkpoint+1:end,:);
trunc2_sex=sex(checkpoint+1:end,:);
svmstruct=svmtrain(trunc1_train,trunc1_sex);
% 使用svmtrain进行训练,得到训练后的结构svmstruct,在预测时使用
group=svmclassify(svmstruct,trunc2_train);
% 对于未知的测试集进行分类预测
count=0
%统计准确率
for i=1:(size(data,1)-checkpoint)
if(group(i,1)==sex(i+checkpoint,1))
count=count+1;
end
end
count/(size(data,1)-checkpoint)
准确率69.87%
分享ppt
分享一次以“神经网络”为主题的ppt，ppt比较长，但主要讲清楚两件事：
①神经网络的基本原理
定义函数->进行训练->确定最优值
②优化算法
ReLU激发函数、交叉熵、动量方法、mini-batch、动态调整学习速率等。
总结
收获
入门了图像处理这一领域，对边缘检测、霍夫变换、滤波等概念有了一个初步了解；
入门了神经网络，知道了其基本原理以及优化算法。
我之所以选这门课，是因为我感觉现在神经网络是未来的趋势，选课前我给自己定的目标就是如果将来面试官问道神经网络的问题我能够说出一些东西来。所以自己也比较重视对基本算法的掌握情况。
遗憾
本课程比较大的遗憾是没有独立的贡献出python代码。本来计划对于SMO算法的SVM程序再进行一下PCA降维，但最终没有实现。自己以前从来没有接触过python，所以比较吃力。不过这也算收获之一吧：接触了Python这一非常具有潜力的语言。
尽管matlab没有开源，无法用它来做这个项目，它也有自己的优势：它有很多分析神经网络的性能的工具，非常适合去真正研究一个神经网络的性质，而非仅仅把它作为一个工具来使用；也正是因为这样点学术界用matlab较多而工业界用python较多。既然它们都有各自的优势，做工程项目完全可以先用MATLAB来分析学习函数、传递函数、网络层数等对于特定问题的影响，把这些确定后再用其他语言构造神经网络。
我觉得matlab的主要优势在于参考资料比较多而且非常准确，语法简单粗暴，而且matlab是面向过程的，在代码量不大的情况下更加清晰明了。而且如果运行matlab的脚本，运行结束后各个变量的值都是保存在workspace里面的，运行结束之后还可以查看各个变量的值，方便调试。但如果在命令行运行python脚本的话，运行结束之后内存空间就被回收了。另外matlab里面的矩阵都可以按照表格样式查看，而python没有这么方便。
尽管python也有大量模块代码，但毕竟是面向对象，对我这种基础不好的人来说还是需要时间的。而写Python可能会出现一些令人抓狂的小问题，例如有一次我运行别人的代码老是失败，折腾了好久才发现是因为我自己的TensorFlow版本过低，还有有时候安装了第三方库还要在/etc/bash.bashrc里面改环境变量，而matlab不需要担心这些问题。
但是一旦有工程需要，包含大量自行设计的模型，就会明白python是多么方便省事了。语言自身的灵活性给自行设计模型提供了极大的方便，这一点matlab能做到，但是很费事。而且做大工程，多人协作等，python比matlab有更多优势。
还有自己和其他同学交流地比较少，我觉得我自己搞得这些东西，例如霍夫变换还有pca，是完全可以集成到现有的代码里面的，但是因为自己水平太低，也没找到人合作，所以都没有成功。总之，以后还要多多努力。