MATLAB: Does train function return neural network with the weights of last epoch or the weights of the model with the smallest loss of all epochs

epocherroriterationlast epochMATLABneural networkreturn neural networkreturn weightsshallowStatistics and Machine Learning Toolboxstoppingtraintraingdtrainingweight

I use the training function "traingd" to train a shallow neural network by using:

[trainedNet,tr] = train(net,X,T)

When the training is done, to which epoch/iteration of the training does the neural network, which is returned as trainedNet by the "train" function, correspond?

Is it the network with the weights of the last epoch of the training before stopping or is it the network with the smallest training error which occured at a certain epoch during the training?

Thanks in advance!

I use MATLAB 2013 and MATLAB 2015 with the neural network toolbox.

Best Answer

You can refer to Early Stopping & Improve Shallow Neural Network Generalization and Avoid Overfitting documentation.

According to it "When the validation error increases for a specified number of iterations (net.trainParam.max_fail), the training is stopped, and the weights and biases at the minimum of the validation error are returned."

For more information on the "net.trainParam.max_fail" parameter, enter the following in command window:

help nnparam.max_fail