2019 年 3 月

使用 PyTorch 的類神經迴歸

[圖 1 使用 PyTorch 示範執行的類神經迴歸

了解資料

``````0.00632, 18.00, 2.310, 0, 0.5380, 6.5750, 65.20
4.0900, 1, 296.0, 15.30, 396.90, 4.98, 24.00
``````

[圖 2 部分波士頓區域房屋資料集

示範程式

``````# boston_dnn.py
# Boston Area House Price dataset regression
# Anaconda3 5.2.0 (Python 3.6.5), PyTorch 1.0.0
import numpy as np
import torch as T  # non-standard alias
# ------------------------------------------------------------
def accuracy(model, data_x, data_y, pct_close):
n_items = len(data_y)
X = T.Tensor(data_x)  # 2-d Tensor
Y = T.Tensor(data_y)  # actual as 1-d Tensor
oupt = model(X)       # all predicted as 2-d Tensor
pred = oupt.view(n_items)  # all predicted as 1-d
n_correct = T.sum((T.abs(pred - Y) < T.abs(pct_close * Y)))
result = (n_correct.item() * 100.0 / n_items)  # scalar
return result
# ------------------------------------------------------------
class Net(T.nn.Module):
def __init__(self):
super(Net, self).__init__()
self.hid1 = T.nn.Linear(13, 10)  # 13-(10-10)-1
self.hid2 = T.nn.Linear(10, 10)
self.oupt = T.nn.Linear(10, 1)
T.nn.init.xavier_uniform_(self.hid1.weight)  # glorot
T.nn.init.zeros_(self.hid1.bias)
T.nn.init.xavier_uniform_(self.hid2.weight)
T.nn.init.zeros_(self.hid2.bias)
T.nn.init.xavier_uniform_(self.oupt.weight)
T.nn.init.zeros_(self.oupt.bias)
def forward(self, x):
z = T.tanh(self.hid1(x))
z = T.tanh(self.hid2(z))
z = self.oupt(z)  # no activation, aka Identity()
return z
# ------------------------------------------------------------
def main():
# 0. Get started
print("\nBoston regression using PyTorch 1.0 \n")
T.manual_seed(1);  np.random.seed(1)
train_file = ".\\Data\\boston_train.txt"
test_file = ".\\Data\\boston_test.txt"
usecols=range(0,13), dtype=np.float32)
usecols=[13], dtype=np.float32)
usecols=range(0,13), dtype=np.float32)
usecols=[13], dtype=np.float32)
# 2. Create model
print("Creating 13-(10-10)-1 DNN regression model \n")
net = Net()  # all work done above
# 3. Train model
net = net.train()  # set training mode
bat_size = 10
loss_func = T.nn.MSELoss()  # mean squared error
optimizer = T.optim.SGD(net.parameters(), lr=0.01)
n_items = len(train_x)
batches_per_epoch = n_items // bat_size
max_batches = 1000 * batches_per_epoch
print("Starting training")
for b in range(max_batches):
curr_bat = np.random.choice(n_items, bat_size,
replace=False)
X = T.Tensor(train_x[curr_bat])
Y = T.Tensor(train_y[curr_bat]).view(bat_size,1)
oupt = net(X)
loss_obj = loss_func(oupt, Y)
loss_obj.backward()
optimizer.step()
if b % (max_batches // 10) == 0:
print("batch = %6d" % b, end="")
print("  batch loss = %7.4f" % loss_obj.item(), end="")
net = net.eval()
acc = accuracy(net, train_x, train_y, 0.15)
net = net.train()
print("  accuracy = %0.2f%%" % acc)
print("Training complete \n")
# 4. Evaluate model
net = net.eval()  # set eval mode
acc = accuracy(net, test_x, test_y, 0.15)
print("Accuracy on test data = %0.2f%%" % acc)
# 5. Save model - TODO
# 6. Use model
raw_inpt = np.array([[0.09266, 34, 6.09, 0, 0.433, 6.495, 18.4,
5.4917, 7, 329, 16.1, 383.61, 8.67]], dtype=np.float32)
norm_inpt = np.array([[0.000970, 0.340000, 0.198148, -1,
0.098765, 0.562177, 0.159629, 0.396666, 0.260870, 0.270992,
0.372340, 0.966488, 0.191501]], dtype=np.float32)
X = T.Tensor(norm_inpt)
y = net(X)
print("For a town with raw input values: ")
for (idx,val) in enumerate(raw_inpt[0]):
if idx % 5 == 0: print("")
print("%11.6f " % val, end="")
print("\n\nPredicted median house price = \$%0.2f" %
(y.item()*10000))
if __name__=="__main__":
main()
``````

將資料載入記憶體

``````train_x = np.loadtxt(train_file, delimiter="\t",
usecols=range(0,13), dtype=np.float32)
usecols=[13], dtype=np.float32)
usecols=range(0,13), dtype=np.float32)
usecols=[13], dtype=np.float32)
``````

定義類神經網路

Net 類別的向前函式會定義圖層如何計算輸出。示範使用 tanh （雙曲線正切） 啟用兩個隱藏的層和輸出層上的無法啟動：

``````def forward(self, x):
z = T.tanh(self.hid1(x))
z = T.tanh(self.hid2(z))
z = self.oupt(z)
return z
``````

PyTorch 是在相當低的層級的抽象概念的方式運作，因為有許多您可以使用的另一種設計模式。例如，而不是定義含有 __init__ 和轉送函式，以及使用 net 然後具現化類別 Net = Net()，您可以使用循序的函式，就像這樣：

``````net = T.nn.Sequential(
T.nn.Linear(13,10),
T.nn.Tanh(),
T.nn.Linear(10,10),
T.nn.Tanh(),
T.nn.Linear(10,1))
``````

定型模型

``````net = net.train()  # Set training mode
bat_size = 10
loss_func = T.nn.MSELoss()  # Mean squared error
optimizer = T.optim.SGD(net.parameters(), lr=0.01)
n_items = len(train_x)
batches_per_epoch = n_items // bat_size
max_batches = 1000 * batches_per_epoch
``````

PyTorch 有兩種模式： 定型和評估。預設模式是訓練，但在我看來，最好明確地將設定模式。（通常稱為 「 迷你批次 」） 的批次大小是超參數。針對迴歸問題，表示平方的誤差最常見的損失函數。隨機梯度下降 (SGD) 演算法是最基本的技巧，並在許多情況下還會使用 Adam 演算法提供更好的結果。

``````for b in range(max_batches):
curr_bat = np.random.choice(n_items, bat_size,
replace=False)
X = T.Tensor(train_x[curr_bat])
Y = T.Tensor(train_y[curr_bat]).view(bat_size,1)
oupt = net(X)
loss_obj = loss_func(oupt, Y)
optimizer.step()     # Update weights and biases
``````

``````if b % (max_batches // 10) == 0:
print("batch = %6d" % b, end="")
print("  batch loss = %7.4f" % loss_obj.item(), end="")
net = net.eval()
acc = accuracy(net, train_x, train_y, 0.15)
net = net.train()
print("  accuracy = %0.2f%%" % acc)
``````

"/ / 」 運算子是在 Python 中的整數除法。然後再呼叫程式定義的精確度函式，示範會設定成評估版模式網路。技術上來說，這並非必要，是因為定型和評估模式只讓不同的結果如果網路使用中輟或圖層的批次正規化。

評估與使用定型的模型

``````net = net.eval()  # set eval mode
acc = accuracy(net, test_x, test_y, 0.15)
print("Accuracy on test data = %0.2f%%" % acc)
``````

Eval 函式會傳回它套用所在之模型的參考它可能已經呼叫沒有指派陳述式。

``````raw_inpt = np.array([[0.09266, 34, 6.09, 0, 0.433, 6.495, 18.4,
5.4917, 7, 329, 16.1, 383.61, 8.67]], dtype=np.float32)
norm_inpt = np.array([[0.000970, 0.340000, 0.198148, -1,
0.098765, 0.562177, 0.159629, 0.396666, 0.260870, 0.270992,
0.372340, 0.966488, 0.191501]], dtype=np.float32)
``````

``````...
X = T.Tensor(norm_inpt)
y = net(X)
print("Predicted = \$%0.2f" % (y.item()*10000))
if __name__=="__main__":
main()
``````

總結

PyTorch 程式庫會比其他替代方案 TensorFlow、 Keras 和 CNTK，特別是與範例程式碼較不成熟。但在我的同事之間 PyTorch 的使用非常快速地成長。我相信若要繼續這項趨勢，而且高品質的範例會越來越多您可以使用。

Dr。James McCaffrey適用於在美國華盛頓州雷德蒙的 Microsoft Research他參與開發數種主要的 Microsoft 產品包括 Internet Explorer 和 Bing。Dr。McCaffrey 要聯絡jamccaff@microsoft.com