The growing inputs method calculates the correlation of every input with every output in the data set. It starts with a neural network that only contains the most correlated input and calculates the selection error for that model. It keeps adding the most correlated variables until the selection error increases.

The objective of model selection is to find the network architecture with the best generalization properties, that is, that which minimizes the error on the selected instances of the data set (the selection error).

How to Choose a Machine Learning Model – Some Guidelines

1. Collect data.
2. Check for anomalies, missing data and clean the data.
3. Perform statistical analysis and initial visualization.
4. Build models.
5. Check the accuracy.
6. Present the results.

An easy guide to choose the right Machine Learning algorithm

1. Size of the training data. It is usually recommended to gather a good amount of data to get reliable predictions.
2. Accuracy and/or Interpretability of the output.
3. Speed or Training time.
4. Linearity.
5. Number of features.

Model selectionis possible using only a subset of your training data, thus saving computational resources(relative ranking-hypothesis) Large overparameterized neural networkscan generalize surprisingly well(double descent)

Two frequent problems in the design of a neural network are called underfitting and overfitting. The best generalization is achieved by using a model whose complexity is the most appropriate to produce an adequate fit of the data. To illustrate underfitting and overfitting, consider the following data set.

Highly overparameterized neural networks can display strong generalization performance, even on small datasets. This is certainly a bold claim, and I suspect many of you are shaking your heads right now.