Introduction

**Chapter 1:** Setting the Stage

**Chapter 2:** Probability

**Chapter 3:** More Probability

**Chapter 4:** Statistics

**Chapter 5:** Linear Algebra

**Chapter 6:** More Linear Algebra

**Chapter 7:** Differential Calculus

**Chapter 8:** Matrix Calculus

**Chapter 9:** Data Flow in Neural Networks

**Chapter 10:** Backpropagation

**Chapter 11:** Gradient Descent

**Appendix:** Going Further

# Math for Deep Learning

Download Chapter 11: GRADIENT DESCENT

__You can find the book's source code on GitHub.__

Deep learning is everywhere, making this powerful driver of AI something more STEM professionals need to know. Learning which library commands to use is one thing, but to truly understand the discipline, you need to grasp the mathematical concepts that make it tick. This book will give you a working knowledge of topics in probability, statistics, linear algebra, and differential calculus – the essential math needed to make deep learning comprehensible, which is key to practicing it successfully.

Each of the four subfields are contextualized with Python code and hands-on, real-world examples that bridge the gap between pure mathematics and its applications in deep learning. Chapters build upon one another, with foundational topics such as Bayes’ theorem followed by more advanced concepts, like training neural networks using vectors, matrices, and derivatives of functions. You’ll ultimately put all this math to use as you explore and implement deep learning algorithms, including backpropagation and gradient descent – the foundational algorithms that have enabled the AI revolution.

You’ll learn:

- The rules of probability, probability distributions, and Bayesian probability
- The use of statistics for understanding datasets and evaluating models
- How to manipulate vectors and matrices, and use both to move data through a neural network
- How to use linear algebra to implement principal component analysis and singular value decomposition
- How to apply improved versions of gradient descent, like RMSprop, Adagrad and Adadelta

Once you understand the core math concepts presented throughout this book through the lens of AI programming, you’ll have foundational know-how to easily follow and work with deep learning.

"Ronald T. Kneusel has written a handy and compact guide to the mathematics of deep learning. It will be a well-worn reference for equations and algorithms for the student, scientist, and practitioner of neural networks and machine learning. Complete with equations, figures and even sample code in Python, this book is a wonderful mathematical introduction for the reader."

**—David S. Mazel, Senior Engineer, Regulus-Group**

"What makes *Math for Deep Learning* a stand-out, is that it focuses on providing a sufficient mathematical foundation for deep learning, rather than attempting to cover all of deep learning, and introduce the needed math along the way. Those eager to master deep learning are sure to benefit from this foundation-before-house approach."

**—Ed Scott, Ph.D., Solutions Architect & IT Enthusiast**