An Introduction to Deep Learning

February 1, 2021 - 2 minute read - Category: Overview - Tags: Neural networks , Backpropogation


There are many fantastic, free resources for understanding the basics of deep learning. In particular, I suggest watching the following four short videos by Grant Sanderson (3blue1brown), who has a YouTube channel teaching mathematical concepts through visual animations. I’m not including a lecture video from my course, because anything I can put together on the basics of neural nets is not going to contribute much given how much great material is already out there.

What is a neural network?

Gradient descent


Backpropogation calculus

References for Lecture 3: An Introduction to Deep Learning

The reference section below outlines some other highly informative references, which I very much recommend checking out for a more complete understanding of the basics of deep learning. Start with the Nielsen book if you’re new to deep learning. If you have some basic familiarity already, the Goodfellow, Bengio, and Courville book is the classic reference, and it’s great to learn the perspectives of the authors, who are pioneers of the field. The third book that I very highly recommend checking out walks you through implementing deep learning methods in PyTorch. Unlike the vast majority of resources cited in this knowledge base, it is not an open source resource, but the guidance it gives on PyTorch is worth paying for. I also cite some blog posts. Colah’s Blog has some classic posts that are very well-known in the deep learning world, and The Decade in Review is a nice overview of some of the most important advances in deep learning during the 2010s, many of which are covered in this knowlede base.

Academic Publications

Web Sources