1、改进1：层数改变
通过修改rbm的层数，可以看到重构误差是变小了，但是，我发现只是修改rbm，并没有影响到最终的accuracy，而且更加诡异的是，这个accuracy，只从我修改了数据集以来，无论怎么修改，都是同一个数，0.391999，即使是同一个数据集，多次运行，结果也不可能一直不变的。
因此，针对数据不变，也许是我的突破点。
'rbm_layers', '52,80,52,3'Training layer 1...
2018-01-05 14:25:33.455536: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu
\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1030] Found device 0 wit
h properties:
name: GeForce GT 720M major: 3 minor: 5 memoryClockRate(GHz): 0.758
pciBusID: 0000:01:00.0
totalMemory: 2.00GiB freeMemory: 1.93GiB
2018-01-05 14:25:33.456106: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu
\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1120] Creating TensorFlo
w device (/device:GPU:0) -> (device: 0, name: GeForce GT 720M, pci bus id: 0000:
01:00.0, compute capability: 3.5)
Tensorboard logs dir for this run is C:\Users\RoFun\.yadlt\logs/run38
Reconstruction loss: 39.7324: 10%|█▎ | 1/10 [00:01<00:09, 1.07s/it
Reconstruction loss: 17.6186: 10%|█▎ | 1/10 [00:01<00:16, 1.80s/it
Reconstruction loss: 17.6186: 20%|██▌ | 2/10 [00:01<00:07, 1.11it/
Reconstruction loss: 17.5831: 20%|██▌ | 2/10 [00:02<00:09, 1.20s/i
Reconstruction loss: 17.5831: 30%|███▉ | 3/10 [00:02<00:05, 1.25it
Reconstruction loss: 17.5797: 30%|███▉ | 3/10 [00:03<00:07, 1.02s/
Reconstruction loss: 17.5797: 40%|█████▏ | 4/10 [00:03<00:04, 1.31
Reconstruction loss: 17.5798: 40%|█████▏ | 4/10 [00:03<00:05, 1.06
Reconstruction loss: 17.5798: 50%|██████▌ | 5/10 [00:03<00:03, 1.3
Reconstruction loss: 17.5818: 50%|██████▌ | 5/10 [00:04<00:04, 1.1
Reconstruction loss: 17.5818: 60%|███████▊ | 6/10 [00:04<00:02, 1.
Reconstruction loss: 17.5843: 60%|███████▊ | 6/10 [00:05<00:03, 1.
Reconstruction loss: 17.5843: 70%|█████████ | 7/10 [00:05<00:02, 1
Reconstruction loss: 17.585: 70%|█████████▊ | 7/10 [00:05<00:02,
Reconstruction loss: 17.585: 80%|███████████▏ | 8/10 [00:05<00:01,
Reconstruction loss: 17.5822: 80%|██████████▍ | 8/10 [00:06<00:01,
Reconstruction loss: 17.5822: 90%|███████████▋ | 9/10 [00:06<00:00,
Reconstruction loss: 17.581: 90%|████████████▌ | 9/10 [00:06<00:00
Reconstruction loss: 17.581: 100%|█████████████| 10/10 [00:06<00:00
, 1.44it/s]
2018-01-05 14:25:42.874094: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu
\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1120] Creating TensorFlo
w device (/device:GPU:0) -> (device: 0, name: GeForce GT 720M, pci bus id: 0000:
01:00.0, compute capability: 3.5)
2018-01-05 14:25:43.029413: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu
\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1120] Creating TensorFlo
w device (/device:GPU:0) -> (device: 0, name: GeForce GT 720M, pci bus id: 0000:
01:00.0, compute capability: 3.5)
2018-01-05 14:25:43.171274: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu
\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1120] Creating TensorFlo
w device (/device:GPU:0) -> (device: 0, name: GeForce GT 720M, pci bus id: 0000:
01:00.0, compute capability: 3.5)
Training layer 2...
2018-01-05 14:25:43.590959: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu
\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1120] Creating TensorFlo
w device (/device:GPU:0) -> (device: 0, name: GeForce GT 720M, pci bus id: 0000:
01:00.0, compute capability: 3.5)
Tensorboard logs dir for this run is C:\Users\RoFun\.yadlt\logs/run39
Reconstruction loss: 0.203016: 10%|█▏ | 1/10 [00:00<00:06, 1.48it/s
Reconstruction loss: 0.0966681: 10%|█ | 1/10 [00:01<00:11, 1.28s/it]
Reconstruction loss: 0.0966681: 20%|██▏ | 2/10 [00:01<00:05, 1.56it/
Reconstruction loss: 0.0584871: 20%|██▏ | 2/10 [00:01<00:07, 1.05it/
Reconstruction loss: 0.0584871: 30%|███▎ | 3/10 [00:01<00:04, 1.58it
Reconstruction loss: 0.0406971: 30%|███▎ | 3/10 [00:02<00:06, 1.10it
Reconstruction loss: 0.0406971: 40%|████▍ | 4/10 [00:02<00:04, 1.47i
Reconstruction loss: 0.030652: 40%|████▊ | 4/10 [00:03<00:05, 1.20i
Reconstruction loss: 0.030652: 50%|██████ | 5/10 [00:03<00:03, 1.50
Reconstruction loss: 0.0244582: 50%|█████▌ | 5/10 [00:03<00:03, 1.26
Reconstruction loss: 0.0244582: 60%|██████▌ | 6/10 [00:03<00:02, 1.5
Reconstruction loss: 0.0202696: 60%|██████▌ | 6/10 [00:04<00:03, 1.3
Reconstruction loss: 0.0202696: 70%|███████▋ | 7/10 [00:04<00:01, 1.
Reconstruction loss: 0.0172196: 70%|███████▋ | 7/10 [00:05<00:02, 1.
Reconstruction loss: 0.0172196: 80%|████████▊ | 8/10 [00:05<00:01, 1
Reconstruction loss: 0.0149137: 80%|████████▊ | 8/10 [00:05<00:01, 1
Reconstruction loss: 0.0149137: 90%|█████████▉ | 9/10 [00:05<00:00,
Reconstruction loss: 0.0131996: 90%|█████████▉ | 9/10 [00:06<00:00,
Reconstruction loss: 0.0131996: 100%|██████████| 10/10 [00:06<00:00,
1.52it/s]
2018-01-05 14:25:51.593593: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu
\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1120] Creating TensorFlo
w device (/device:GPU:0) -> (device: 0, name: GeForce GT 720M, pci bus id: 0000:
01:00.0, compute capability: 3.5)
2018-01-05 14:25:51.740685: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu
\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1120] Creating TensorFlo
w device (/device:GPU:0) -> (device: 0, name: GeForce GT 720M, pci bus id: 0000:
01:00.0, compute capability: 3.5)
2018-01-05 14:25:51.877104: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu
\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1120] Creating TensorFlo
w device (/device:GPU:0) -> (device: 0, name: GeForce GT 720M, pci bus id: 0000:
01:00.0, compute capability: 3.5)
Training layer 3...
2018-01-05 14:25:52.294727: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu
\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1120] Creating TensorFlo
w device (/device:GPU:0) -> (device: 0, name: GeForce GT 720M, pci bus id: 0000:
01:00.0, compute capability: 3.5)
Tensorboard logs dir for this run is C:\Users\RoFun\.yadlt\logs/run40
Reconstruction loss: 0.180896: 10%|█▏ | 1/10 [00:00<00:06, 1.45it/s
Reconstruction loss: 0.0777469: 10%|█ | 1/10 [00:01<00:11, 1.33s/it]
Reconstruction loss: 0.0777469: 20%|██▏ | 2/10 [00:01<00:05, 1.50it/
Reconstruction loss: 0.0398954: 20%|██▏ | 2/10 [00:01<00:07, 1.03it/
Reconstruction loss: 0.0398954: 30%|███▎ | 3/10 [00:01<00:04, 1.54it
Reconstruction loss: 0.0232394: 30%|███▎ | 3/10 [00:02<00:05, 1.18it
Reconstruction loss: 0.0232394: 40%|████▍ | 4/10 [00:02<00:03, 1.57i
Reconstruction loss: 0.0152276: 40%|████▍ | 4/10 [00:03<00:04, 1.27i
Reconstruction loss: 0.0152276: 50%|█████▌ | 5/10 [00:03<00:03, 1.59
Reconstruction loss: 0.0103914: 50%|█████▌ | 5/10 [00:03<00:03, 1.29
Reconstruction loss: 0.0103914: 60%|██████▌ | 6/10 [00:03<00:02, 1.5
Reconstruction loss: 0.00745202: 60%|██████ | 6/10 [00:04<00:03, 1.33
Reconstruction loss: 0.00745202: 70%|███████ | 7/10 [00:04<00:01, 1.5
Reconstruction loss: 0.00557388: 70%|███████ | 7/10 [00:05<00:02, 1.3
Reconstruction loss: 0.00557388: 80%|████████ | 8/10 [00:05<00:01, 1.
Reconstruction loss: 0.00433528: 80%|████████ | 8/10 [00:05<00:01, 1.
Reconstruction loss: 0.00433528: 90%|█████████ | 9/10 [00:05<00:00, 1
Reconstruction loss: 0.00336939: 90%|█████████ | 9/10 [00:06<00:00, 1
Reconstruction loss: 0.00336939: 100%|█████████| 10/10 [00:06<00:00, 1
.54it/s]
2018-01-05 14:26:00.234303: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu
\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1120] Creating TensorFlo
w device (/device:GPU:0) -> (device: 0, name: GeForce GT 720M, pci bus id: 0000:
01:00.0, compute capability: 3.5)
2018-01-05 14:26:00.382128: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu
\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1120] Creating TensorFlo
w device (/device:GPU:0) -> (device: 0, name: GeForce GT 720M, pci bus id: 0000:
01:00.0, compute capability: 3.5)
2018-01-05 14:26:00.521913: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu
\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1120] Creating TensorFlo
w device (/device:GPU:0) -> (device: 0, name: GeForce GT 720M, pci bus id: 0000:
01:00.0, compute capability: 3.5)
Training layer 4...
2018-01-05 14:26:00.940003: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu
\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1120] Creating TensorFlo
w device (/device:GPU:0) -> (device: 0, name: GeForce GT 720M, pci bus id: 0000:
01:00.0, compute capability: 3.5)
Tensorboard logs dir for this run is C:\Users\RoFun\.yadlt\logs/run41
Reconstruction loss: 0.376031: 10%|█▏ | 1/10 [00:00<00:07, 1.26it/s
Reconstruction loss: 0.3409: 10%|█▍ | 1/10 [00:01<00:12, 1.40s/it
Reconstruction loss: 0.3409: 20%|██▊ | 2/10 [00:01<00:05, 1.43it/
Reconstruction loss: 0.305654: 20%|██▍ | 2/10 [00:01<00:07, 1.00it/
Reconstruction loss: 0.305654: 30%|███▌ | 3/10 [00:01<00:04, 1.50it
Reconstruction loss: 0.276197: 30%|███▌ | 3/10 [00:02<00:06, 1.15it
Reconstruction loss: 0.276197: 40%|████▊ | 4/10 [00:02<00:03, 1.54i
Reconstruction loss: 0.251135: 40%|████▊ | 4/10 [00:03<00:04, 1.25i
Reconstruction loss: 0.251135: 50%|██████ | 5/10 [00:03<00:03, 1.56
Reconstruction loss: 0.229447: 50%|██████ | 5/10 [00:03<00:03, 1.31
Reconstruction loss: 0.229447: 60%|███████▏ | 6/10 [00:03<00:02, 1.
Reconstruction loss: 0.210518: 60%|███████▏ | 6/10 [00:04<00:02, 1.
Reconstruction loss: 0.210518: 70%|████████▍ | 7/10 [00:04<00:01, 1
Reconstruction loss: 0.19387: 70%|█████████ | 7/10 [00:05<00:02, 1
Reconstruction loss: 0.19387: 80%|██████████▍ | 8/10 [00:05<00:01,
Reconstruction loss: 0.17917: 80%|██████████▍ | 8/10 [00:05<00:01,
Reconstruction loss: 0.17917: 90%|███████████▋ | 9/10 [00:05<00:00,
Reconstruction loss: 0.166124: 90%|██████████▊ | 9/10 [00:06<00:00,
Reconstruction loss: 0.166124: 100%|███████████| 10/10 [00:06<00:00,
1.59it/s]
2018-01-05 14:26:08.502946: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu
\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1120] Creating TensorFlo
w device (/device:GPU:0) -> (device: 0, name: GeForce GT 720M, pci bus id: 0000:
01:00.0, compute capability: 3.5)
2018-01-05 14:26:08.647751: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu
\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1120] Creating TensorFlo
w device (/device:GPU:0) -> (device: 0, name: GeForce GT 720M, pci bus id: 0000:
01:00.0, compute capability: 3.5)
2018-01-05 14:26:08.783165: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu
\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1120] Creating TensorFlo
w device (/device:GPU:0) -> (device: 0, name: GeForce GT 720M, pci bus id: 0000:
01:00.0, compute capability: 3.5)
Start deep belief net finetuning...
2018-01-05 14:26:10.667883: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu
\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1120] Creating TensorFlo
w device (/device:GPU:0) -> (device: 0, name: GeForce GT 720M, pci bus id: 0000:
01:00.0, compute capability: 3.5)
Tensorboard logs dir for this run is C:\Users\RoFun\.yadlt\logs/run42
Accuracy: 0.438: 10%|██▌ | 1/10 [00:01<00:11, 1.23s/i
Accuracy: 0.438: 10%|██▌ | 1/10 [00:02<00:20, 2.29s/i
Accuracy: 0.438: 20%|█████▏ | 2/10 [00:02<00:09, 1.15
Accuracy: 0.438: 20%|█████▏ | 2/10 [00:03<00:14, 1.76
Accuracy: 0.438: 30%|███████▊ | 3/10 [00:03<00:08, 1.
Accuracy: 0.438: 30%|███████▊ | 3/10 [00:04<00:10, 1.
Accuracy: 0.438: 40%|██████████▍ | 4/10 [00:04<00:06,
Accuracy: 0.438: 40%|██████████▍ | 4/10 [00:05<00:08,
Accuracy: 0.438: 50%|█████████████ | 5/10 [00:05<00:05
Accuracy: 0.438: 50%|█████████████ | 5/10 [00:06<00:06
Accuracy: 0.438: 60%|███████████████▌ | 6/10 [00:06<00
Accuracy: 0.438: 60%|███████████████▌ | 6/10 [00:08<00
Accuracy: 0.438: 70%|██████████████████▏ | 7/10 [00:08
Accuracy: 0.438: 70%|██████████████████▏ | 7/10 [00:09
Accuracy: 0.438: 80%|████████████████████▊ | 8/10 [00:
Accuracy: 0.438: 80%|████████████████████▊ | 8/10 [00:
Accuracy: 0.438: 90%|███████████████████████▍ | 9/10 [
Accuracy: 0.438: 90%|███████████████████████▍ | 9/10 [
Accuracy: 0.438: 100%|█████████████████████████| 10/10
[00:11<00:00, 1.13s/it]
2018-01-05 14:26:26.744554: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu
\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1120] Creating TensorFlo
w device (/device:GPU:0) -> (device: 0, name: GeForce GT 720M, pci bus id: 0000:
01:00.0, compute capability: 3.5)
Test set accuracy: 0.3919999897480011
请按任意键继续. . .2、查看GitHub时，看到被人是这样调用，虽然他的还是出错
import tensorflow as tf
from yadlt.models.boltzmann import dbn
from tensorflow.examples.tutorials.mnist import input_data
# load the data
mnist = input_data.read_data_sets("tensorflow_learning/MNIST_data/", one_hot = True)
x_train = mnist.train.images
y_train = mnist.train.labels
x_val = mnist.validation.images
y_val = mnist.validation.labels
# create a DBN
my_DBN = dbn.DeepBeliefNetwork(rbm_layers = [50, 30, 20],
finetune_loss_func = 'softmax_cross_entropy',
finetune_dropout = 1,
finetune_learning_rate = 0.001,
finetune_act_func = tf.nn.sigmoid,
finetune_num_epochs = 20,
finetune_batch_size = 20,
do_pretrain = True)
my_DBN.pretrain(x_train, x_val)
my_DBN.build_model(784, 10)
my_DBN._train_model(x_train, y_train, x_val, y_val)