http://blog.****.net/pipisorry/article/details/52250983
选择合适的estimator
通常机器学习最难的一部分是选择合适的estimator,不同的estimator适用于不同的数据集和问题。
sklearn官方文档提供了一个图[flowchart],可以快速地根据你的数据和问题选择合适的estimator,单击相应的区域还可以获得更具体的内容。
代码中我一般这么写
def gen_estimators(): ''' List of the different estimators. ''' estimators = [ # ('Lasso regression', linear_model.Lasso(alpha=0.1), True), ('Ridge regression', linear_model.Ridge(alpha=0.1), True), # ('Hinge regression', linear_model.Hinge(), True), # ('LassoLars regression', linear_model.LassoLars(alpha=0.1), True), ('OrthogonalMatchingPursuitCV regression', linear_model.OrthogonalMatchingPursuitCV(), True), ('BayesianRidge regression', linear_model.BayesianRidge(), True), ('PassiveAggressiveRegressor regression', linear_model.PassiveAggressiveRegressor(), True), ('HuberRegressor regression', linear_model.HuberRegressor(), True), # ('LogisticRegression regression', linear_model.LogisticRegression(), True), ] return estimators
然后如下遍历算法
def cross_validate(): for name, clf, flag in gen_estimators(): ) clf.fit(x_train, y_train) print(name, '\n', clf.coef_) # scores = cross_val_score(clf, x, y, cv=5, scoring='roc_auc') y_score = clf.predict(x_test) y_score = np.select([y_score < 0.0, y_score > 1.0, True], [0.0, 1.0, y_score]) scores = metrics.roc_auc_score(y_true=[1.0 if _ > 0.0 else 0.0 for _ in y_test], y_score=y_score) ) X_train.shape, y_train.shape ((90, 4), (90,)) X_test.shape, y_test.shape ((60, 4), (60,)) clf = svm.SVC(kernel='linear', C=1).fit(X_train, y_train) clf.score(X_test, y_test) 0.96...
sklearn交叉验证
scores , scoring=rocAucScorer)
自定义CV策略
(cv是整数的话默认使用KFold):
>>> n_samples = iris.data.shape[0] >>> cv = cross_validation.ShuffleSplit(n_samples, n_iter=3, test_size=0.3, random_state=0) >>> cross_validation.cross_val_score(clf, iris.data, iris.target, cv=cv) array([ 0.97..., 0.97..., 1. ])
另一个接口cross_val_predict ,可以返回每个元素作为test set时的确切预测值(只有在CV的条件下数据集中每个元素都有唯一预测值时才不会出现异常),进而评估estimator:
>>> predicted = cross_validation.cross_val_predict(clf, iris.data, iris.target, cv=10)
>>> metrics.accuracy_score(iris.target, predicted)
0.966...
Scikit-learn:并行调参Grid Search
Grid Search: Searching for estimator parameters
scikit-learn中提供了pipeline(for estimator connection) & grid_search(searching best parameters)进行并行调参
如使用scikit-learn做文本分类时:vectorizer取多少个word呢?预处理时候要过滤掉tf>max_df的words,max_df设多少呢?tfidftransformer只用tf还是加idf呢?classifier分类时迭代几次?学习率怎么设?……
“循环一个个试”,这就是grid search要做的基本东西。
from: http://blog.****.net/pipisorry/article/details/52250983
ref: [scikit-learn User Guide]
[Model selection and evaluation]
[3.1. Cross-validation: evaluating estimator performance]*
[3.2. Grid Search: Searching for estimator parameters]*
[Parameter estimation using grid search with cross-validation*]
[Sample pipeline for text feature extraction and evaluation*]