Submitted by [deleted] t3_zk5ga6 in deeplearning
MIKOLAJslippers t1_izy9s51 wrote
Reply to comment by Melodic-Oil-1971 in does anyone know how to build neural network from scratch? by [deleted]
So you will need to implement that maths in your chosen language (easiest would be python and numpy as the syntax is almost the same as I shared). That’s the forward pass from inputs to outputs. You will also need to initialise the weight matrices w1 and w2 to something. Do you have any pretrained weights you can test it with? You may also need to add biases after the matmuls depending on the brief. Usually the case but not necessarily essential to make it train.
Presumably you will also need to then train your network so it’ll get a bit more tricky. You’ll need to implement a loss function based on error between the outputs and some target variable. Once you have the loss you can then use chain rule back through the network to get the delta w (weight gradients) for each weight (w1 and w2 and also any biases if you add those). You’ll then update your weights using some update rule which is usually just multiplying the weight gradients by the learning rate (usually denoted alpha).
Is any of this helpful? Which bit do you still not understand?
Melodic-Oil-1971 t1_izya5wt wrote
Thank you that was helpful
Viewing a single comment thread. View all comments