pytorch中的dropout方法可以用來刪除一些不必要的特征值,但是每次Dropout的時(shí)候Dropout掉的參數(shù)可能都不一樣,那么pytorch 中nn.Dropout如何優(yōu)化呢?接下來這篇文章告訴你。
結(jié)論
Pytorch的nn.Dropout在每次被調(diào)用時(shí)dropout掉的參數(shù)都不一樣,即使是同一次forward也不同。
如果模型里多次使用的dropout的dropout rate大小相同,用同一個(gè)dropout層即可。
如代碼所示:
import torch
import torch.nn as nn
class MyModel(nn.Module):
def __init__(self):
super(MyModel, self).__init__()
self.dropout_1 = nn.Dropout(0.5)
self.dropout_2 = nn.Dropout(0.5)
def forward(self, input):
# print(input)
drop_1 = self.dropout_1(input)
print(drop_1)
drop_1 = self.dropout_1(input)
print(drop_1)
drop_2 = self.dropout_2(input)
print(drop_2)
if __name__ == '__main__':
i = torch.rand((5, 5))
m = MyModel()
m.forward(i)
結(jié)果如下:
*\python.exe */model.pytensor([[0.0000, 0.0914, 0.0000, 1.4095, 0.0000],[0.0000, 0.0000, 0.1726, 1.3800, 0.0000],[1.7651, 0.0000, 0.0000, 0.9421, 1.5603],[1.0510, 1.7290, 0.0000, 0.0000, 0.8565],[0.0000, 0.0000, 0.0000, 0.0000, 0.0000]])tensor([[0.0000, 0.0000, 0.4722, 1.4095, 0.0000],[0.0416, 0.0000, 0.1726, 1.3800, 1.3193],[0.0000, 0.3401, 0.6550, 0.0000, 0.0000],[1.0510, 1.7290, 1.5515, 0.0000, 0.0000],[0.6388, 0.0000, 0.0000, 1.0122, 0.0000]])tensor([[0.0000, 0.0000, 0.4722, 0.0000, 1.2689],[0.0416, 0.0000, 0.0000, 1.3800, 0.0000],[0.0000, 0.0000, 0.6550, 0.0000, 1.5603],[0.0000, 0.0000, 1.5515, 1.4596, 0.0000],[0.0000, 0.0000, 0.0000, 0.0000, 0.0000]])Process finished with exit code 0
以上就是pytorch 中nn.Dropout如何優(yōu)化的全部內(nèi)容,希望能給大家一個(gè)參考,也希望大家多多支持W3Cschool。