Tag Archives: programming

Data science: An introduction to neural networks

Artificial neural networks can be thought of as a set of interconnected nodes somewhat similar to brain neurons in nature. Computer neural networks can be used to analyze or categorize items from data, such as recognizing hand-written digits (see the image above) or faces in images.

Here is how I started teaching myself the basics of computer neural networks. The key information resources and tools I used are listed below.

I did most of my learning on a MacBook Pro laptop running macOS High Sierra. I also did some exploring of programming exercises using a VMWARE Fusion virtual copy of the Raspbian Linux distribution. I believe all the required tools can also be used on Windows.

The Python programing examples in the books require that you install the numpy, matplotlib, and tensorflow Python modules. It is also possible to use the Thonny Python IDE to run the programming examples.

The first half of the book by Tariq Rashid is a technical overview of a simple algorithm that is used to process data by a neural network. In this part of the book there are many nice diagrams that show how a computer neural network works. I went through this part slowly and from this I got the theory behind a neural networking algorithm that optimizes the network based on mathematical gradient descent techniques. The application is to recognize numbers contained in a set of images from the MNIST database of handwritten digits. The neural network takes as input a set of 784 pixels (which represent a 28X28 pixel image) and the output is the digit contained in the image. There is a single hidden layer between the input and output nodes of the network.

In the second half of the Rashid book, a series of Python programs are created in Jupyter notebooks. The end result is a Python program that faithfully implements the  neural network algorithm presented in the first half of the book. I used copy and paste to also create a single Python script from the notebooks that I could modify and run in IDLE.

The first part of the book by Michael Taylor describes a neural networking algorithm very similar to that of the Rashid book, so I skimmed that part. In the second half of the Taylor book there is a description of a Python program that uses the TensorFlow module to implement the algorithm. TensorFlow is an open source machine learning framework. It allows you to code many variations of neural networks, but I found it a little too general-purpose for someone just getting started with neural networks.

Sample projects

Once I created the Python program from the Rashid book, I modified it to handle another application. In this application there was a single input of a number “x” between 0.0 and Pi. The output was the value of sin(x). You can see from the following plot how good the result was:

Neural network sin(x) 0 to pi

In order to test the model further, I changed the range of inputs to 0.0 to 1.5*pi. As you can see, in this case the model does not do well at all! Applying neural networks requires both skill and creativity.

Neural network sin(x) 0.0 to 3*pi/2

What’s next

After finishing my introductory education on data science and neural networks I searched for appropriate follow-on educational opportunities available online. I found a set of classes (Data Science Professional Certificate) given by Rafael Irizarry of Harvard University, sponsored by edX, a MOOC provider.

I have already completed three of nine classes, and I am very happy with what I have seen so far. When I have completed the entire series I will have a certificate in data science.

For more information

These are the key information resources and tools I used in my early education on this topic.

 Make Your Own Neural Network by  Tariq Rashid

Neural Networks – A Visual Introduction for Beginners by Michael Taylor

Anaconda Navigator IDE, which supports Jupyter notebooks

IDLE with Python 3.7