有時候你會發現，讀書時讀太久把書背太死，考試時題型變化一點點可能就不會寫；對應到機器學習的情境下，就是較複雜的模型訓練了很久之後，雖然能把訓練集的模樣學得更透徹，但是到測試集上的表現就會炸掉，這樣的情況稱為過擬合(overfitting)。相反地，如果模型太簡單，可能連訓練集都學不好，這種情況則稱為欠擬合(underfitting)。一個基本的範例如下，其中資料是由三次多項式加上雜訊所產生，而我們分別使用一次式（太簡單）、三次式（剛好）以及十五次式（太複雜）來進行擬合：

Sometimes you may find that if you study something for too long and memorize it too rigidly, even a small change in the exam questions can make you unable to answer. In machine learning, this is like when a more complex model is trained for a long time. It learns the training set very well, but it performs badly on the test set. We call this overfitting. On the other hand, if the model is too simple, it may not even learn the training set well. We call this underfitting. A simple example is shown below. The data comes from a cubic polynomial plus some noise. We fit the data using a linear function (too simple), a cubic function (just right), and a degree-15 polynomial (too complex):

import numpy as np
import matplotlib.pyplot as plt
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression

np.random.seed(42)
X = np.linspace(0, 10, 20).reshape(-1, 1)
y = 0.1 * X[:, 0] ** 3 - 2 * X[:, 0] ** 2 + 10 * X[:, 0] + 3 + np.random.randn(20) * 3

X_plot = np.linspace(0, 10, 100).reshape(-1, 1)

_, axes = plt.subplots(1, 3)
titles = ['Underfitting (degree 1)', 'Good Fit (degree 3)', 'Overfitting (degree 15)']

for i, degree in enumerate([1, 3, 15]):
    poly = PolynomialFeatures(degree=degree)
    X_poly = poly.fit_transform(X)
    X_plot_poly = poly.transform(X_plot)
    
    model = LinearRegression()
    model.fit(X_poly, y)
    y_plot = model.predict(X_plot_poly)
    
    axes[i].scatter(X, y, s=50, label='Training data')
    axes[i].plot(X_plot, y_plot, 'r-', linewidth=2, label=f'Degree {degree}')
    axes[i].set_title(titles[i], fontsize=14)
    axes[i].set_xlabel('x')
    axes[i].set_ylabel('y')
    axes[i].legend()

plt.show()

因此，為了保護模型的穩定，防止模型過擬合，我們可以在模型的訓練過程當中，不時的將目前的模型用驗證集來測試；萬一模型在驗證集上的表現變差了，就代表過擬合可能已經發生了，你可以準備提前停止訓練(early stopping)。除了透過驗證集來監控過擬合之外，也有許多其他的手段可以避免過擬合的發生，例如正規化(regularization)，或者減少模型的複雜度等等。

So, to keep the model stable and stop it from overfitting, we can test the current model on the validation set from time to time during training. If the model's result on the validation set gets worse, this may mean overfitting has started. Then you can stop the training early. We call this early stopping.Besides using the validation set to watch for overfitting, there are also many other ways to avoid overfitting, such as regularization, or making the model less complex.