Training neural networks: loss functions, SGD, learning rates, Adam and RMSprop, dropout and schedulers

This week’s topics

Training neural networks: loss functions, SGD, learning rates, Adam and RMSprop, dropout and schedulers

1. Training neural networks: loss functions, SGD, learning rates, Adam and RMSprop, dropout and schedulers
2. Deep vs. shallow networks

What you will be able to do

• 2.3 Explain the importance of loss functions in providing an objective for the training of neural networks.
• 2.4 Analyse and compare common optimisation strategies for neural network training.
• 2.5 Evaluate the advantages and disadvantages of deep and shallow neural network architectures.

Activity

Seminar, guided reading and a short written task: explain the importance of loss functions in providing an objective for the training of neural networks.

Study materials