Neural networks are just some complex formulas. Not that it involves some complex and fancy mathing it is just multiplication and addition with applied non-linearity. I would suggest making some simple neural networks for like digit recognition using MNIST database. After you understand all the maths than you can go deeper.