There has been an incredible amount of advancement with regard to machine learning during these last few years. Problems which used to take research teams months or years can now be easily implemented by a skilled programmer using machine learning techniques. This is the first in a series of articles that should point you in the right direction as to how to start using machine learning, and in particular, deep learning techniques as easily as possible.
Getting the right tools
Before you get involved with machine learning, you should get the right tools for the job. There are several good packages available which make implementing and training deep neural networks easy, and these packages tend to have two things in common. They all run on Linux, more specifically Ubuntu, and they all support CUDA acceleration via graphics cards. So before you get too deep into the subject, you owe it to yourself to have a good processor with ample memory, and a good Nvidia graphics card. The more memory on your graphics card, the better, since that will allow you to train on larger datasets and have larger networks.
The three big ones
Even with Google’s recently released TensorFlow, there are still three big ones already available.
Torch – This is my favorite tool for deep learning, so I will list it first. The documentation is excellent, and there are ample tutorials in place that will teach you how to do tasks such as import data, preprocess data, create a neural network, train it, and anything else can imagine. Torch has a command line interface, and thankfully, if you install iTorch, you can have a notebook interface which makes writing and debugging code a breeze. Anything from viewing images, to visualizing weights is extremely easy with the iTorch notebook interface. And best of all, the backend functions are written in C++ and many functions have CUDA accelerated variants. The only thing I dislike about it is that you need to write in LUA, but honestly, it is very to learn. There is a fun tutorial called Learn Lua in 15 Minutes that I would highly recommend. If you’ve ever planning on doing anything custom or anything super interested with deep neural networks, I would recommend Torch.
Caffe – This tool is written in C++, is integrated into Nvidia’s DIGITs package, and is very performant. If you just want to do classification problems, it may be sufficient for you just to download Nvidia’s DIGIT package and use that. With this package, you set up your neural network with a text file describing the network. Then you can train the network with data. I personally do not use Caffe much due to lack of documentation, awkwardness of handling non-image data, and the amount of work involved in doing anything custom. Remember, this is written in C++, which was once my favorite language, but it does take a lot of code and knowledge to get interesting things done in Caffe.
Theano – Theano is a python package that many people who already know Python prefer. It is easy to setup and train neural networks with Theano, but it is not well integrated with CUDA, which means that it can be significantly slower than Caffe or Torch. If you really, absolutely must use Python, then you can use Theano. But for any serious project or deep network, I would have to recommend you use something more performant.
Setting up Linux
For deep learning projects, I use Ubuntu 14.04, because I know both CUDA and Torch are supported on it. Unfortunately, it isn’t the latest version of Ubuntu, so if you have a newer computer, you may have to battle a little bit to install drivers for your audio chip and network chip. But assuming you’re able to get all the drivers to work, having Ubuntu 14.04 is a great start to getting involved in deep learning. Next, you will need to install the CUDA toolkit.
Setting up CUDA
At the time of this writing, CUDA Toolkit 7.5 is the latest, and there is a version for Ubuntu 14.04 and Ubuntu 15.04. No matter what, make sure you follow all the instructions in the NVIDIA CUDA Getting Started Guide for Linux. You will need to set some environment variables after installation to make everything work properly. I always download the NVIDIA CUDA SDK with all the examples so I make sure they all compile and can run before moving on to install Torch or Caffe. No, you don’t have to make sure each and every one run, but spot check a few of the simulation examples. If they have trouble compiling or running, something may be amiss with your installation.
Setting up Torch
For the foreseeable future, my articles will be written using Torch code for examples. To install torch, I recommend starting here. Once torch is installed, the next thing you should do is install iTorch. iTorch is a very nice graphical interface for you to write and run Torch code. I highly recommend using iTorch because it makes debugging much easier. To do so, go here and scroll down to where it says ‘Installing iTorch.’ Once you’re up and running with linux, CUDA, torch and iTorch, you’ll be ready to start tackling machine learning problems like a pro!
If you’ve setup your machine and installed torch and iTorch, you are ready to get started! Check out my next tutorial to get you started learning simple convolutional networks!