بسم الله الرحمن الرحيم
Alhamdulillaah, I am learning Machine Learning and I am mostly looking at Neural Networks. I started off with modelling the logical OR using a perceptron. By Grace of الله, I was able to move to the Multi Layer Perceptron (MLP), which can handle much more complicated problems, like regression and classification of "non linear" data.
There's a term called Linear Separability, which is the peculiar characteristic of a dataset which can be separated by a line (or a plane/hyperplane if there are more than 2 features in the input). Most data is non linear, such that even for modelling the logical XOR, the perceptron proves to be inadequate.
Using the MLP, I was, by the Grace of الله, able to train a neural network to predict correct outputs for XOR. Then I tried the regression problem. I attempted to train the MLP to regress the data which was generated using a sinusoidal function. الحمد لله, I learnt quite a few things from it.
One is that as more nodes are added in the hidden layer, the network gets closer to the data.
The thin green curve is the actual function which the MLP tries to regress. The thin red curve is the MLP's prediction with two nodes in the hidden layer. The thick blue curve is the MLP with 5 nodes in the hidden layer. I used only one hidden layer for both MLPs. So it can be seen that the red curve, though it does not predict the data as well as the blue curve, especially on the left side, it still manages to generate values which more or less represent the average values of the data.
