I would like to measure models performance by looking for AUC or Accuracy. In the grid search I get results with residual deviance
,how can I tell h2o deep learning grid to have AUC instead of residual deviance and present the results as atable like the one attached below ?
我想通过寻找AUC或精确度来测量模型的性能。在网格搜索中,我得到的结果带有剩余偏差,如何告诉h2o深度学习网格具有AUC而不是剩余偏差,并将结果显示为如下所附的可读结果?
train <- read.table(text = "target birds wolfs snakes
0 9 7 a
0 8 4 b
1 2 8 c
1 2 3 a
1 8 3 a
0 1 2 a
0 7 1 b
0 1 5 c
1 9 7 c
1 8 7 c
0 2 7 b
1 2 3 b
1 6 3 c
0 1 1 a
0 3 9 a
1 1 1 b ",header = TRUE)
trainHex <- as.h2o(train)
g <- h2o.grid("deeplearning",
hyper_params = list(
seed = c(123456789,12345678,1234567),
activation = c("Rectifier", "Tanh", "TanhWithDropout", "RectifierWithDropout", "Maxout", "MaxoutWithDropout")
),
reproducible = TRUE,
x = 2:4,
y = 1,
training_frame = trainHex,
validation_frame = trainHex,
epochs = 50,
)
g
model_ids <- g@summary_table
model_ids<-as.data.frame(model_ids)
The results table that I got:
我得到的结果表:
Hyper-Parameter Search Summary: ordered by increasing residual_deviance
activation seed model_ids residual_deviance
1 Maxout 12345678 Grid_DeepLearning_train_model_R_1483217086840_112_model_10 0.07243775676256235
2 Maxout 1234567 Grid_DeepLearning_train_model_R_1483217086840_112_model_16 0.10060885040861599
3 MaxoutWithDropout 123456789 Grid_DeepLearning_train_model_R_1483217086840_112_model_5 0.1706496158406441
4 Maxout 123456789 Grid_DeepLearning_train_model_R_1483217086840_112_model_4 0.17243125875659948
5 Tanh 123456789 Grid_DeepLearning_train_model_R_1483217086840_112_model_1 0.18326527198894926
6 Tanh 12345678 Grid_DeepLearning_train_model_R_1483217086840_112_model_7 0.18763395264761593
7 Tanh 1234567 Grid_DeepLearning_train_model_R_1483217086840_112_model_13 0.18791531211136187
8 TanhWithDropout 123456789 Grid_DeepLearning_train_model_R_1483217086840_112_model_2 0.19808063817007837
9 TanhWithDropout 12345678 Grid_DeepLearning_train_model_R_1483217086840_112_model_8 0.19815190962052193
10 TanhWithDropout 1234567 Grid_DeepLearning_train_model_R_1483217086840_112_model_14 0.19832946889767458
11 Rectifier 123456789 Grid_DeepLearning_train_model_R_1483217086840_112_model_0 0.20679125165086842
12 MaxoutWithDropout 1234567 Grid_DeepLearning_train_model_R_1483217086840_112_model_17 0.21971759565380736
13 RectifierWithDropout 123456789 Grid_DeepLearning_train_model_R_1483217086840_112_model_3 0.22337599298253263
14 MaxoutWithDropout 12345678 Grid_DeepLearning_train_model_R_1483217086840_112_model_11 0.22440661112729862
15 RectifierWithDropout 1234567 Grid_DeepLearning_train_model_R_1483217086840_112_model_15 0.2284671685474275
16 RectifierWithDropout 12345678 Grid_DeepLearning_train_model_R_1483217086840_112_model_9 0.23163744415703522
17 Rectifier 1234567 Grid_DeepLearning_train_model_R_1483217086840_112_model_12 0.2516917276707789
18 Rectifier 12345678 Grid_DeepLearning_train_model_R_1483217086840_112_model_6 0.2642221616447725
1 个解决方案
#1
3
You can do this with h2o.getGrid()
. Following on from your example code:
可以使用h2o.getGrid()来实现这一点。根据示例代码:
g_rmse <- h2o.getGrid(g@grid_id, "rmse")
g_rmse #Output it
I've chosen root-MSE there. AUC is not available for your sample data: it has to be a binomial classification, and you are doing a regression.
我选择root-MSE那里。AUC不能用于您的样本数据:它必须是二项分类,并且您正在进行回归。
The reason you are doing a regression is your y
contains 0 and 1, so H2O has guessed it is numeric. You need to use as.factor()
on that column, just after uploading it into H2O.
进行回归的原因是y包含0和1,所以H2O认为它是数值。您需要在该列上使用as.factor(),就在将其上传到H2O之后。
train <- ...
trainHex <- as.h2o(train)
trainHex[,1] = as.factor(trainHex[,1]) #Add this
g <- ...
Then you can do this:
然后你可以这样做:
g_auc <- h2o.getGrid(g@grid_id, "auc", decreasing = TRUE)
g_auc
I've set it to decreasing=TRUE
so that the best AUC is at the top.
我将它设为递减=TRUE以便最好的AUC在顶部。
#1
3
You can do this with h2o.getGrid()
. Following on from your example code:
可以使用h2o.getGrid()来实现这一点。根据示例代码:
g_rmse <- h2o.getGrid(g@grid_id, "rmse")
g_rmse #Output it
I've chosen root-MSE there. AUC is not available for your sample data: it has to be a binomial classification, and you are doing a regression.
我选择root-MSE那里。AUC不能用于您的样本数据:它必须是二项分类,并且您正在进行回归。
The reason you are doing a regression is your y
contains 0 and 1, so H2O has guessed it is numeric. You need to use as.factor()
on that column, just after uploading it into H2O.
进行回归的原因是y包含0和1,所以H2O认为它是数值。您需要在该列上使用as.factor(),就在将其上传到H2O之后。
train <- ...
trainHex <- as.h2o(train)
trainHex[,1] = as.factor(trainHex[,1]) #Add this
g <- ...
Then you can do this:
然后你可以这样做:
g_auc <- h2o.getGrid(g@grid_id, "auc", decreasing = TRUE)
g_auc
I've set it to decreasing=TRUE
so that the best AUC is at the top.
我将它设为递减=TRUE以便最好的AUC在顶部。