* , the model output
* , the desired target
* , the square of the error,

the LMS algorithm updates the weights as follows:

This update rule minimizes , the square of the error, and is in fact the stochastic gradient descent update for linear regression.

MADALINE

MADALINE (Many ADALINE) is a three-layer (input, hidden, output), fully connected, feedforward neural network