An Introduction to Deep Learning
Topics
This post covers the first lecture in the course: “An Introduction to Deep Learning.”
This lecture will begin by setting the stage for why deep learning is necessary. When curating data at scale, there are generally two approaches: humandesigned rules and deep learning. While we’ll see throughout the course that the judicious use of rules has its place in many big data endeavors, generally the performance of rulebased methods is disappointing in comparison to deep learning. The lecture will provide a couple of case studies of deep learning applied to economic data. We will then turn to an overview of neural networks.
For those who need to brush up on deep learning methods, the online text Neural Networks and Deep Learning provides an introduction to the main concepts and how they relate. Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville is the standard reference in the field. It is best suited to those who have some basic familiarity with deep learning. Part I provides a solid introduction to the concepts in applied math that are required to understand deep learning (students in the course should be familiar with this material, but the book is a good refresher). I recommend the book Deep Learning with PyTorch to anyone who does not already have extensive familiarity with PyTorch. There are also numerous lectures available for free online.
Lecture Video
References Cited in Lecture 1: Introduction to Deep Learning
Academic Papers
Other Resources
Textbooks

Ian Goodfellow and Yoshua Bengio and Aaron Courville. Deep Learning, 2016, MIT Press

Stevens, Eli, Luca Antiga, and Thomas Viehmann. Deep learning with PyTorch. Manning Publications Company, 2020.
Other Resources and Videos

This blog is a classic for neural networks background: https://colah.github.io/

A basic introduction to neural nets: https://www.3blue1brown.com/topics/neuralnetworks

Weights & biases tutorials: https://wandb.ai/site/tutorials
Code Bases
 PyTorch official examples: https://github.com/pytorch/examples. Often very helpful to review relevant examples once one has a use case in mind.
Image Source: https://www.freecodecamp.org/news/wanttoknowhowdeeplearningworksheresaquickguideforeveryone1aedeca88076/