多層感知器(Multilayer perceptron, MLP)的一層，可以表示為 ŷ = f(xTW + b)，其中的 x 為輸入向量，W 為權重矩陣，b 為偏差向量，f 為 activation function，ŷ 為輸出向量。並且由於堆疊多層時，每一層的每個輸出都是完全連接到下一層的每個輸入，所以這樣的每一層又叫做全連接層(fully connected layer)。多層感知器或者其他各種類神經網路的訓練，則首先需要使用損失函數(loss function)評估網路算出來的 ŷ 和標準答案 y 之間的差異，透過你所選用的工具（例如 PyTorch 或者 TensorFlow）進行反向傳播求出梯度以後，再使用優化器(optimizer)來更新每個 W 和 b。如此反覆進行之後，通常就可以訓練出恰當的類神經網路。

One layer of a multilayer perceptron (MLP) can be written as ŷ = f(xTW + b). Here, x is the input vector, W is the weight matrix, b is the bias vector, f is the activation function, and ŷ is the output vector. When we stack many layers, every output of one layer is connected to every input of the next layer, so each layer like this is also called a fully connected layer. To train an MLP, or any other kind of neural network, we first use a loss function to measure the difference between the network's output ŷ and the correct answer y. Then, we use the tool you choose (such as PyTorch or TensorFlow) to do backpropagation and get the gradients. After that, we use an optimizer to update every W and b. If we repeat this again and again, we can usually train a good neural network.

對於 activation function、損失函數以及最佳化方法，可以依照你要處理的問題特性，或者實際效果來選擇。常用的 activation function 有 sigmoid、ReLU、tanh，損失函數則有常用於回歸問題的 mean squared error (MSE)，以及常用於分類問題的 cross entropy 等可以選擇。最佳化方法則通常有 SGD (stochastic gradient descent)、Momentum、AdaGrad、RMSProp，以及 Adam 等方式，若無特殊需求，且在可以選擇的情況下，通常你可以先試試看 Adam。

You can choose the activation function, the loss function, and the optimizer based on the problem you want to solve, or based on how well they work in practice. Common activation functions are sigmoid, ReLU, and tanh. For loss functions, you can use mean squared error (MSE), which is common for regression problems, and cross entropy, which is common for classification problems. Common optimizers are SGD (stochastic gradient descent), Momentum, AdaGrad, RMSProp, and Adam. If you have no special needs and you can choose, you can usually try Adam first.

若要使用 MLP 解決問題，有很多工具可以幫忙，常見或曾經常見的有 PyTorch、TensorFlow，以及 Chainer 等等，不過這些工具為了能讓你做更自由的調整，所以在真正的類神經網路本身和資料的事前處理，都有許多工作要自己做，因此以下先使用 scikit-learn 的 neural_network.MLPClassifier 來示範 MLP 的使用，範例的資料集是 Wine Data Set：

There are many tools that can help you solve problems with an MLP. Common ones, or ones that used to be common, are PyTorch, TensorFlow, and Chainer. But these tools let you make very flexible adjustments, so you have to do a lot of work yourself, both on the neural network itself and on the data preprocessing. So, below we will first use neural_network.MLPClassifier from scikit-learn to show how to use an MLP. The dataset in this example is the Wine Data Set:

import numpy as np
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
from sklearn.neural_network import MLPClassifier

dataset = load_wine()
print('Data shapes:', dataset.data.shape, dataset.target.shape)

X_train, X_test, y_train, y_test = train_test_split(dataset.data, dataset.target)
print('Training data shapes:', X_train.shape, y_train.shape)
print('Test data shapes:', X_test.shape, y_test.shape)

model = MLPClassifier(hidden_layer_sizes=(256, 256))
model.fit(X_train, y_train)
pred = model.predict(X_test)
print('Accuracy: {:.2f}%'.format(100*np.mean(pred == y_test)))

hidden = np.maximum(X_test @ model.coefs_[0] + model.intercepts_[0], 0)
hidden = np.maximum(hidden @ model.coefs_[1] + model.intercepts_[1], 0)
pred = np.argmax(hidden @ model.coefs_[2] + model.intercepts_[2], axis=1)

print('Accuracy: {:.2f}%'.format(100*np.mean(pred == y_test)))

在上述範例中，

In the example above,

如果你對超參數或特徵等方面的選擇不太有想法，或者想要在二維平面上，看看不同超參數在不同資料上的視覺化效果，可以前往 TensorFlow Playground 網站嘗試。

If you have no idea how to choose hyperparameters or features, or you want to see how different hyperparameters work on different data on a 2D plane, you can try the TensorFlow Playground website.

如前所述，若使用 PyTorch 等工具來訓練類神經網路，則為了能讓你做更自由的調整，有許多超參數都需要一一明確指定，以下是把前一個範的預測工作，改用 PyTorch 來進行：

As we said before, when you use tools like PyTorch to train a neural network, you need to set many hyperparameters one by one, so that you can make flexible adjustments. Below, we do the same prediction task as the previous example, but with PyTorch:

import torch
import torch.nn as nn
from torch.utils.data import DataLoader, TensorDataset
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split

device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')

dataset = load_wine()
print('Data shapes:', dataset.data.shape, dataset.target.shape)

X_train, X_test, y_train, y_test = train_test_split(dataset.data, dataset.target)
print('Training data shapes:', X_train.shape, y_train.shape)
print('Test data shapes:', X_test.shape, y_test.shape)

X_train = torch.FloatTensor(X_train)
X_test = torch.FloatTensor(X_test)
y_train = torch.LongTensor(y_train)

train_loader = DataLoader(
	TensorDataset(X_train, y_train), batch_size=128, shuffle=True
)

model = nn.Sequential(
	nn.Linear(13, 256),
	nn.ReLU(),
	nn.Linear(256, 256),
	nn.ReLU(),
	nn.Linear(256, 3)
).to(device)
optim = torch.optim.Adam(model.parameters())
criterion = nn.CrossEntropyLoss()

model.train()
for _ in range(200):
	for X_batch, y_batch in train_loader:
		loss = criterion(model(X_batch.to(device)), y_batch.to(device))
		optim.zero_grad()
		loss.backward()
		optim.step()

model.eval()
pred = model(X_test.to(device)).argmax(dim=1).detach().cpu().numpy()
print('Accuracy: {:.2f}%'.format(100 * (pred == y_test).mean()))

在上述範例中，

In the example above,