Welcome to Part 11 of the Python Plays: Grand Theft Auto V tutorial series, where we're working on creating a self-driving car in the game.

Leading up to this point, we've built a training dataset that consists of 80x60 resized game imagery data, along with keyboard inputs for A,W, and D (left, forward, and right respectively).

Next, we need to create and train a neural network for this task. For visual data, it tends to be the case that a Convolutional Neural Network makes the most sense. From here, we still have tons of choices, with layers, nodes per layer, activation functions, and more. I elected to just use AlexNet for the model.

We're going to be making use of TFLearn (TensorFlow abstraction library). If you want to use raw TensorFlow, or maybe Keras, or maybe Theano, or who knows what else, go for it! If you need to learn more about how TFLearn works, check out the TFLearn tutorial.

Now let's define the model. We should probably save this in a new file, we'll call it alexnet.py

# alexnet.py

import tflearn
from tflearn.layers.conv import conv_2d, max_pool_2d
from tflearn.layers.core import input_data, dropout, fully_connected
from tflearn.layers.estimator import regression
from tflearn.layers.normalization import local_response_normalization

def alexnet(width, height, lr):
    network = input_data(shape=[None, width, height, 1], name='input')
    network = conv_2d(network, 96, 11, strides=4, activation='relu')
    network = max_pool_2d(network, 3, strides=2)
    network = local_response_normalization(network)
    network = conv_2d(network, 256, 5, activation='relu')
    network = max_pool_2d(network, 3, strides=2)
    network = local_response_normalization(network)
    network = conv_2d(network, 384, 3, activation='relu')
    network = conv_2d(network, 384, 3, activation='relu')
    network = conv_2d(network, 256, 3, activation='relu')
    network = max_pool_2d(network, 3, strides=2)
    network = local_response_normalization(network)
    network = fully_connected(network, 4096, activation='tanh')
    network = dropout(network, 0.5)
    network = fully_connected(network, 4096, activation='tanh')
    network = dropout(network, 0.5)
    network = fully_connected(network, 3, activation='softmax')
    network = regression(network, optimizer='momentum',
                         loss='categorical_crossentropy',
                         learning_rate=lr, name='targets')

    model = tflearn.DNN(network, checkpoint_path='model_alexnet',
                        max_checkpoints=1, tensorboard_verbose=2, tensorboard_dir='log')

    return model

Now we can work on our train_model.py:

# train_model.py

import numpy as np
from alexnet import alexnet

WIDTH = 80
HEIGHT = 60
LR = 1e-3
EPOCHS = 8
MODEL_NAME = 'pygta5-car-{}-{}-{}-epochs.model'.format(LR, 'alexnetv2',EPOCHS)

model = alexnet(WIDTH, HEIGHT, LR)

Setup the training data:

train_data = np.load('training_data_v2.npy')

train = train_data[:-500]
test = train_data[-500:]

X = np.array([i[0] for i in train]).reshape(-1,WIDTH,HEIGHT,1)
Y = [i[1] for i in train]

test_x = np.array([i[0] for i in test]).reshape(-1,WIDTH,HEIGHT,1)
test_y = [i[1] for i in test]

Now we can actually train the model with:

model.fit({'input': X}, {'targets': Y}, n_epoch=EPOCHS, validation_set=({'input': test_x}, {'targets': test_y}), 
    snapshot_step=500, show_metric=True, run_id=MODEL_NAME)

# tensorboard --logdir=foo:C:/Users/H/Desktop/ai-gaming/log

model.save(MODEL_NAME)

If you want to see your model's progress and some more information on it, you can use the commented-out tensorboard command in your cmd.exe.

Once you're done training, you're ready for the next step. I suggest you get about 5-15 epochs to start. I went with 8. If you want more information on "how many epochs should you do," see the video, same with if you want more information on Tensor Board.