What You'll Build in This Series
This course takes you from zero to understanding neural networks at a deep level by coding everything from scratch. You'll start with raw Python (no libraries) then move to NumPy for performance and real-world applications. This foundational approach aligns with topics covered in the Comprehensive Python Course: From Basics to Advanced Mega Projects and will prepare you for advanced concepts like those in Understanding Introduction to Deep Learning: Foundations, Techniques, and Applications.
Prerequisites
- Programming basics with Object-Oriented Programming (OOP)
- Python fundamentals (lists, loops, functions, f-strings)
- No prior deep learning or math knowledge required , everything will be explained
Why Build From Scratch?
Most deep learning courses hand you pre-built frameworks where you just "plug and play" without understanding why things work. This leads to problems when:
- Solving custom problems that haven't been solved for you
- Moving beyond simple tasks like digit classification
- Needing to fix issues without knowing where to look
The Forward Pass (Simplified)
What looks complex in math notation becomes simple in code:
- Input × Weights → Dot product (
np.dot) - Add Bias →
output + bias - Activation Function →
max(0, output)(ReLU) or softmax - Loss Calculation → Negative log loss
Neural Network Anatomy
Components
- Neurons (blue circles) , processing units
- Connections (orange lines) , unique weights
- Neuron bias , unique to each neuron
- Layers: Input → Hidden(s) → Output
How It Works
Data flows forward through the network, with each layer transforming the information via weighted sums and activation functions. The network learns by adjusting thousands of weights and biases through training.
Getting Started: Code a Single Neuron
inputs = [1.2, 5.1, 2.1] # Output from previous layer's neurons
weights = [3.1, 2.1, 8.7] # Unique weights for each connection
bias = 3 # Bias for this neuron
# Single neuron output calculation
output = inputs[0]*weights[0] + inputs[1]*weights[1] + inputs[2]*weights[2] + bias
print(output) # 35.7
Series Resources
- Free YouTube series covering all content
- Book available at nnfs.io (ebook, softcover, hardcover) for deeper study
- Discord community: discord.gg/sentdex for help and discussion
- Technology stack: Python 3.7.7+, NumPy 1.18.2+, Matplotlib 3.2.1+
What Makes This Approach Unique
- Truly from scratch , no black boxes
- Dual implementation: raw Python + NumPy (see the Python NumPy Tutorial: Arrays, Operations, and Data Science Notebooks for a deeper dive)
- Focus on deep understanding over memorization
- Framework-agnostic knowledge applicable to PyTorch, TensorFlow, or future tools
- All concepts simplified to painfully simple building blocks
Your Learning Journey
Expect to spend multiple sessions across different mediums (video + book) to fully absorb the material. The goal isn't speed, it's true comprehension that lets you solve custom problems. For additional foundational Python practice, refer to the Comprehensive Python Tutorial: Variables, Control Flow, Functions, Classes, and File Handling. As you progress, the Comprehensive Artificial Intelligence Course: AI, ML, Deep Learning & NLP can help you see how these fundamentals scale to real-world AI systems.
What's going on everybody welcome to the Neural Networks from Scratch series where we will be creating creating Neural Networks end-to-end so from...
the Neurons, the Layers, Activation functions, Optimization,... Back Propagation all this stuff were going to be coding
that from scratch in python. Now... everything we do we are going to show first in truly raw python no libraries or no 3rd party libraries
and then we are going to use NumPy for multiple reasons NumPy just makes a lot of sense here. It's a extremely useful library and
it'll cut our lines of a full application down-a-ton it'll make it much faster and NumPy is a great thing to learn, so...
we will show everything from scratch in python first and then we are going to use NumPy. Now... why would anybody wanna do this to themselves? Well, first of all, it's very interesting umm...
the idea is not-even though we effectively we are going to be programming our own Neural Networks framework, it's not really the
purpose here. The purpose is to actually learn how Neural Networks work at a very deep level so that when we go back to whatever Framework
we actually use, be it PyTorch or Tensorflow's Keras or maybe some library that doesn't even exist yet. We actually understand what we are doing. So
from myself when I at least when I learned Deep Learning, it was all- -yes it was hard but at the same time everything was kinda solved for me you know, how
many layers, how many multiple layers, what Activation functions to use but all that stuff was just fed to me I just kind of-okay
and then I just sort of memorized like here's the activation function used for this sort of task But I didn't understand why and
this became a problem when I tried to solve problems that had not yet been solved for me. So classifying images of handwritten
digits? Pretty darn simple. Taking that a step further and classifying images of cats and dogs? Pretty simple.
But then taking that just a tiny step further and classifying images that were instead frames from a video game and trying to map that to actions that I want to take
in the video game. Suddenly, I'm lost. and that's..... ...that's no good and there's no way to really know
where to go next and you can see that there is a problem but actually trying to solve more custom problems is going to require to have a deeper
understanding of how things actually work Unfortunately though deeper understanding Deep Learning can be very very complicated.
So you can just-just a simple forward pass of a Neural Network Neural Network looks extremely daunting. So looking at the calculations
for a Neural Network, it looks pretty confusing fairly quick You've got your input data and every unique input and every unique neuron.
The information coming through has a unique weight associated- with it. Those get summed together per neuron neuron plus a bias. Run through an
activation function and then we do that for every single layer, giving us the output information. From there we wanna calculate a loss
which is a calculation of how wrong the neural Network is, so that hopefully we can fix it. And at the end of that, even though that was just a forward pass
that looks already extremely-extremely daunting Now let's take a look at the exact same formula just in code version. You have all your inputs times your weights (Input x Weights)
And you don't have to follow along perfectly here just look at each element and see if any of these elements look like they are over your head. They really shouldn't be. So inputs times weights, you can make this even simpler by doing just a dot product so 'np.dot'
y1 is just the max of zero and the output We do this for all the layers. Then we have our softmax (activation function) at the very end.
And then the loss turns out to be a negative log (logarithmic) loss due to the nature of neural networks.
So this is the entire forward pass and calculation of loss formula I urge you to look at all of the functions and things that we are doing here to determine if any of these is really over your head because it shouldn't be right?
We've got very simple functions going on here We are calculating a log, sum-if you don't know what a 'log' means btw we're going to explain it. 'Sum', you should know what that means.
'Exponential', again if you don't know what that means we are going to explain but it's very simple. 'Dot Product', again if you don't know what that means no worries we will explain it, very simple.
'Maximum', is just the max whatever two values you pass here. Again very simple. Some more dot products, 'Transpose', again very simple.
If you don't know it we will explain it anyway That's it! None of this is over your head I promise you As far as prerequisites go the only expectation I have from the viewers is that you understand
programming and Object-Oriented Programming (OOP) otherwise, you're going to feel kinda lost. If you're coming from a different programming language you will probably find
python simple You can probably use the programming language that you are familiar with Everything is so low level here that you should be able
to follow along in any other language that you want So feel free to do that, if you wanna do that otherwise, if you wanna follow along in python make sure you know the basics and OOP
I'll put links in the description for both of those If you need to brush up Next, is the version of python
If you follow along with python we are gonna do things like 'fstring' I'm gonna be on Python 3.7 The NumPy version, I can't think of any reason of why it would matter.
I'll put that information in the description just in case any function does change. Like I said, everything's so low level this series should be good for like 10 years.
Let's hope for that :) So those are the prerequisites it's really not much.
I don't expect anybody to have any background knowledge of Deep Learning So if you do know things about deep learning, yes we are going to cover hopefully quickly the fundamentals, just so people understand
like what exactly are we aiming for here? And then the bulk of your understanding of how Neural Networks work is gonna come from us just building these Neural Networks
If things feel a little fuzzy to you it's probably normal, to be honest. I think once you build a Neural Network from scratch
that is all the understanding you are gonna need So you're not expected to know math or anything like that. For me in college, the only math class I took was 'Math Fundamentals'
And I don't think we did any math calculation at all in that class. It was definitely a joke. So if I can do this, I know you can do this too.
So If you wanna brush up on Math There is Khan academy for Linear Algebra and the calculus stuff But I wouldn't even suggest you go through the full series on either of those topics
You can use those to kinda like a spot check issues that you still find confusing With that in mind, this series is also provided in conjunction with the 'Neural Networks from Scratch' book We are going to be covering the same material for the most part.
The book might be a little more verbose. The series is obviously free but the book ha various prices depending on what you want.
The E-book, Softcover, Hardcover. We ship everywhere in the world Access to the book gives you access to the E-book. So whichever version you buy you always have access to the E-book.
That gives you the opportunity to access the google docs draft, currently it's in a draft form at some point, it won't be a draft anymore
but you can highlight, post comments, ask questions inline with the text Also if you're impatient you can access even though its a draft right now it's complete from basically end-to-end
of training the Neural Network and we're doing testing right now. So, we are already obviously quite a bit ahead of where the videos are
so if you're impatient you can also access information earlier with the book I would use the book as either reviews or the videos as reviews So I would maybe read the book before watching the video and then use the video as a review of what I learned in the book or vice versa
This is a topic that you're not gonna blow through this in a weekend. It's gonna require multiple sittings multiple environments and ideally multiple mediums so,
If you're interested in the book you can get that at 'nnfs.io'. So we call these Neural Networks because they look visually like a network. You've got your Neurons which in this case are the blue circles.
They're connected via those orange lines and in this case, we have basically the input layer, two hidden layers of 4 neurons each and then your output layer.
Now data is gonna get past forward through this, so it starts at your input layer, so in this case, we only have two pieces of data they are gonna come in.
That gets past forward to that first hidden layer Then that hidden layer passes data to the second layer and then finally to the output layer where we hope that it will output something that we want, for example:
based on some sensor data maybe we wanna predict Failure or not failure. You could either have one neuron or in this case we have two neurons. So the top neuron might be a failure neuron, the bottom neuron is a not failure neuron.
And depending on which one has the higher value that's the supposed prediction. Now the end goal of neural networks like most machine learning is to take some input data
and produce output data that is desired. In this case, we've got images of cats and dogs. We hope that we can pass it through in pixel form to our neural network
and if its a dog then that final output neuron on top is going to be the strongest, if it's a cat then that final output neuron on the bottom will be the strongest.
And we can do this by tuning the weights and biases So all those unique weights and biases, we do that by tuning those and that is the actual training process.
It's tuning those in such a way so that hopefully, we can take data that this neural network has never seen, give it pictures of cats and dogs that it has never seen
and have it accurately predict those So how and why do neural networks work? Well, if you just look at them and really consider what's going on
every neuron is connected to the subsequent layer of neurons in folds. So each of those orange lines, that connection that's a unique weight
and then every neuron is a unique bias. So what this ends up giving us is a huge number of uniquely tunable parameters
that go into this GIGANTIC function. So for example, with 64 x 3 hidden layers here we have 9164 tunable parameters
in this gigantic function. And each of those parameters impacts the output of the next neurons and so on. And so what we end up having
are these complex relationships that can in theory be mapped. So to me the impressive thing of neural networks is not
necessarily all of those connections its not really complex to understand. What the hard part of neural networks and deep learning is figuring out
how to tune such a thing. Alright! since I think it would be lame to not post any code at all in this first video we are going to begin to code neuron.
But first I wanna go over the version numbers real quick because quite frankly I am gonna forget to put it in the description So I'm using Python 3.7.7
NumPy 1.18.2 Matplotlib 3.2.1 Again all of this should work
very far into the future But just in case not, there are the exact versions if anybody needs to have the exact version to follow along
Now, let's go ahead and begin to code just to say I am using Sublime Text You can use whatever editor you want.
Everyone's got a really strong opinion on editors. I'm gonna be using Sublime text. There might be a time where we use Jupyter Notebook. Who knows, I'm gonna use whatever makes the most sense to me at the time
which may not make sense to you or me later Anyway, continuing on, so every neuron basically- Let's pretend we are coding a neuron that's somewhere in this-
densely connected feed-forward multilayer perceptron model. Using all the big words. Don't worry about knowing what those mean, by the end of this
you will know all of those words. But part the problem with learning deep learning is people use the same 3-4 words for the exact same thing and it can be very daunting. So for now,
we are gonna called this a neuron, it's somewhere in our neural network. Now in this fully connected neural network every neuron has a unique connection
to every single previous neuron. So, let's say there are 3 neurons that are feeding into this neuron that we are gonna build.
So, we don't know much about those neurons but we know that they are outputting some value So first, their output becomes the neurons that we are coding's inputs We are gonna just make up some numbers, we're just gonna say 1.2, 5.1, 2.1
Those are the unique inputs So these are outputs from the three neurons in the previous layer Every unique input is also going to have a unique weight associated with it.
So we are going to say weights and you should know how many weights we are going to have Well since we have three inputs we know we are going to have three weights (3.1, 2.1, 8.7) I'm just making up these numbers.
It's just for, beginning to code how neurons are gonna work So you've got your inputs, your weights and then every unique neuron has a unique bias
So bias equals to 3 So now, the first step to for a neuron is to add up all the inputs times weights plus the bias
So this is relatively simple. In very raw python no loops required at this stage. We're just going to say basically
output so far of this neuron is going to be *look at the screen now* *You need to have some basic knowledge of 'list' here*
That is basically so far... There will be other things that will happen soon enough, but so far that is the output to our neural network
So we'll just print the output, run it. 35.7 If you're new to Sublime Text for some reason 'ctrl + b' to run. But you'll have to set up the build system to run with python.
But I would expect how that people know how to work with a programming language But if not feel free to
either comment below, join us in 'discord.gg/sentdex And we'll be happy to help. Anyway that's it for now
if you've got questions, comments, concerns, whatever moving forward feel free to ask them below otherwise, we're just gonna keep slowly shipping away
at Neural Networks from Scratch As you can see pretty darn simple so far and for the most part, it's just adding a bunch of-
as we've broken it down so far We are gonna break it down to the point where every little additional step is gonna look an awful lot like that
There's a couple of points that may be a little more challenging but the goal is to break it down so much that it is painfully simple
So I will see you guys in the next video :)
You need basic programming knowledge with Object-Oriented Programming (OOP) and Python fundamentals like lists, loops, functions, and f-strings. No prior deep learning or math knowledge is required, as everything is explained from the ground up.
Building from scratch ensures you understand the inner workings of neural networks, enabling you to solve custom problems, debug issues effectively, and move beyond simple tasks. This approach provides framework-agnostic knowledge, so you can adapt to any future tool like PyTorch or TensorFlow with a deeper comprehension.
The series covers neurons (processing units), connections with unique weights, neuron-specific biases, and layers including input, hidden, and output layers. Data flows forward through these components via weighted sums and activation functions, with the network learning by adjusting thousands of weights and biases during training.
The forward pass simplifies complex math into code steps: first, compute the dot product of inputs and weights using np.dot, then add a bias, apply an activation function like ReLU (max(0, output)) or softmax, and finally calculate the loss using negative log loss. This is demonstrated with a single neuron example using Python lists and basic arithmetic.
The series includes a free YouTube series covering all content, a book available at nnfs.io (in ebook, softcover, and hardcover formats) for deeper study, and a Discord community at discord.gg/sentdex for help and discussion. The technology stack uses Python 3.7.7+, NumPy 1.18.2+, and Matplotlib 3.2.1+.
This series is truly from scratch with no black boxes, using dual implementation in raw Python and NumPy for performance. It focuses on deep understanding over memorization, breaking concepts into painfully simple building blocks, and ensures framework-agnostic knowledge applicable to any future deep learning tools.
Expect to spend multiple sessions across different mediums (video and book) to fully absorb the material. The goal is true comprehension that lets you solve custom problems, not speed. For foundational Python practice, refer to the linked comprehensive Python tutorial, and as you advance, explore the linked AI course to see how fundamentals scale to real-world systems.
Keep this summary
Save it to LunaNotes and it becomes a real note in your library — editable, searchable, and ready to turn into flashcards or a diagram. Free to start.
Save to LunaNotesOr summarise for another video.
32K+ people have turned 65K+ videos into notes
“Very useful tool. I'm using it for the last couple weeks and I'm loving it so far. Makes learning on YouTube a great experience.”
Bhai Chrome Web Store reviewThis summary and transcript were automatically generated using AI with the Free YouTube transcript summary tool by LunaNotes.
Related summaries
Understanding Introduction to Deep Learning: Foundations, Techniques, and Applications
Explore the exciting world of deep learning, its techniques, applications, and foundations covered in MIT's course.
AI Fundamentals: Machine Learning, Deep Learning & Neural Networks Explained
This comprehensive lecture covers the core concepts of Artificial Intelligence, from everyday applications like Face ID and recommendations to the technical foundations of Machine Learning, Deep Learning, and Neural Networks. Learn the key differences between supervised, unsupervised, and reinforcement learning, and understand how models are trained and used for predictions.
Comprehensive Python Course: From Basics to Advanced Mega Projects
This extensive Python course covers everything from fundamental programming concepts, data types, and control flow to advanced topics like OOP, file handling, virtual environments, and AI integration. Featuring practical projects including a Jarvis assistant and chatbot, it equips learners with hands-on skills for professional growth and job readiness.
Python Mastery Course: Complete Beginner to Pro Guide (2024)
Learn Python from scratch with this comprehensive guide covering variables, strings, numbers, and essential programming concepts. Master Python for AI, web development, and automation with step-by-step tutorials from an experienced software engineer.
Mastering Sequence Modeling with Recurrent Neural Networks
Explore the concepts of sequence modeling, RNNs, and their applications in machine learning.
Most viewed summaries
A Comprehensive Guide to Using Stable Diffusion Forge UI
Explore the Stable Diffusion Forge UI, customizable settings, models, and more to enhance your image generation experience.
Kolonyalismo at Imperyalismo: Ang Kasaysayan ng Pagsakop sa Pilipinas
Tuklasin ang kasaysayan ng kolonyalismo at imperyalismo sa Pilipinas sa pamamagitan ni Ferdinand Magellan.
Mastering Inpainting with Stable Diffusion: Fix Mistakes and Enhance Your Images
Learn to fix mistakes and enhance images with Stable Diffusion's inpainting features effectively.
Pamamaraan at Patakarang Kolonyal ng mga Espanyol sa Pilipinas
Tuklasin ang mga pamamaraan at patakaran ng mga Espanyol sa Pilipinas, at ang epekto nito sa mga Pilipino.
How to Install and Configure Forge: A New Stable Diffusion Web UI
Learn to install and configure the new Forge web UI for Stable Diffusion, with tips on models and settings.
Found this summary useful?
Take it with you. One click puts it in your own LunaNotes library.
Save to LunaNotes