The Levenberg-Marquardt (LM) algorithm is a popular optimisation technique for training feedforward neural networks. It blends gradient descent and the Gauss-Newton method to adjust weights efficiently.
Strengths:
- Fast convergence
- Effective for small to medium-sized datasets
Limitations:
- High memory usage
- Less scalable for large parameter spaces