Traffic Sign Recognition

Programming Language:

• Python

The goals of this project are the following:

• Load the data set (see below for links to the project data set)
• Explore, summarize and visualize the data set
• Design, train and test a model architecture
• Use the model to make predictions on new images
• Analyze the softmax probabilities of the new images
• Summarize the results with a written report

Step1: Data Set Summary & Exploration

1. A basic summary of the data set.

I use Pickle data load function to read our dataset, and then use basic python function to analyze the dataset. In total, we have:

• Training samples: 34,799
• Validation samples: 4,410
• Testing samples: 12,630
• Each sample (image) has (32, 32, 3) shape.
• We have 43 unique classes/labels:

2. Here is an exploratory visualization of the data set.

Step2: Design and Test a Model Architecture

1. Image data preprocessing

I use two methods to do data augmentation (preprocessing).

1. Randomly rotation: all training images have 50% probability to be rotated within (-30, 30) angle. Rotation is needed considering that in daily life, the camera on a self-driving car may capture a traffice sign with a radom angle. Thus, by doing this help the model has the capability of handling such an application scenario.
2. Randomly crop: all training images have 50% probability to be cropped from (0.8, 1) of width and height. After being cropped, the images will be resized back to 32x32. Cropping is necessary because the captured traffic signs have variant sizes. Thus, using cropping adds more training data. In addition, randomly cropping provides some smaples with width and height shifts.

Here are some processed images:

2. Model architecture

My model consisted of the following layers:

3. Training parameters

I use the following experimental settings to train my model:

• Optimizer: Adam
• Initialial Learning Rate: 0.001
• Batch Size: 128
• Training Epochs: 100
• Loss function: cross entropy.

4. Accuracy

During training, I tracked the validation accuracy and only saved the weights achieved the highest validation accuracy. Within 100 training epochs, my architecture achieves the highest val_acc at 84 epoch. (All details about training process can be found in Cell [10])

• The accuracy on training set: 99.9%
• The accuracy on validation set: 96.2%
• The accuracy on testing set: 93.4%

The followings are the reasons why I chose such a model for this task:

1. In the first convolutional laeyrs, the model focuses on detecting fundamental features, e.g. lines, shapes, or textures. Thus, at the first two blocks (4 convolutional layers), I used two continuous convolutional layers to make sure that the model generates more useful basic features.
2. Following that, a block with one convolutional layer is used to generate semantic information, e.g. arrows, circles, and etc.. Comparing to the basic features, we have more semantic features, thus, more filters were used in this layer.
3. A fully connected layer is used to convert a feature map into a verctor for classification.
4. One Dropout layer is used to avoid overfitting.

Test a Model on New Images

1. Unseen data during traing

Here are five German traffic signs that I found on the web:

2. Prediction results on these new traffic signs.

Here are the results of the prediction:

The model was able to correctly guess 4 of the 5 traffic signs, which gives an accuracy of 80%. We can see it drops a lot comparing to the accuracy on our testing dataset. The reason is three-fold:

1. The training dataset is not balance. For example, class 2 (Speed limit (50km/h)) has 2010 training samples, while class 14 (Stop) only consistes of 330 training samples.
2. The downloaded images are much clearer than the images in our dataset, which may add more noise in the images.
3. 5 images are not enough to evaluate the performance of a trained model.

3. Prediction details

For the first image, the model is 100% sure that it is a “Go straight or right”. It is correctly predict this image.

For the second image, the model is 100% sure that it is a “Speed limit (50km/h)”. It is correctly predict this image.

For the third image, the model is 100% sure that it is a “Yield”, while the ground truth is “Stop”. There are only 690 “Stop” training samples in our dataset, while “Yield” has 1290 training images. This is caused by our un-balance training data.

For the fourth image, the model is 100% sure that it is a “Road work”. It is correctly predict this image.

For the fourth image, the model is 100% sure that it is a “Turn right ahead”. It is correctly predict this image.

Visualizing the model

I visulized the output of the first block. The circle and arrows are very clear.