Practical Deep Neural Networks AI - Best Practices for Gradient Learning

Overview

What you will study

In the Practical Deep Neural Networks AI - Best Practices for Gradient Learning course offered by University Teknologi PETRONAS , we first introduce the methodology of gradient learning and backpropagation and highlight where gradient learning commonly fails. We review common training loss functions and regularization strategies which improve the convergence of gradient learning. With a good understanding of these fundamentals, we will study the motivation and implementation of input, weight and activation normalizations and clipping techniques that have been commonly used to stabilize gradient learning across multiple different network architectures. We will discuss a numerical technique to check gradients to assess the success of gradient learning. Finally, we will study methods to enhance learning convergence through adaptive learning algorithms.

Upon completion of this course, participants will be able to:

• Apply gradient learning best practices to train deep neural networks correctly.

• Improve the performance or robustness of deep neural networks.