Welcome to part 4 of the deep learning basics with Python, TensorFlow, and Keras tutorial series. In this part, what we're going to be talking about is TensorBoard. TensorBoard is a handy application that allows you to view aspects of your model, or models, in your browser.

The way that we use TensorBoard with Keras is via a Keras callback. There are actually quite a few Keras callbacks, and you can make your own. Definitely check the others out: Keras Callbacks. For example, ModelCheckpoint is another useful one. For now, however, we're going to be focused on the TensorBoard callback. We

To begin, we need to add the following to our imports:

from tensorflow.keras.callbacks import TensorBoard

Now we want to make our TensorBoard callback object:

NAME = "Cats-vs-dogs-CNN"

tensorboard = TensorBoard(log_dir="logs/{}".format(NAME))

Eventually, you will want to get a little more custom with your NAME, but this will do for now. So this will save the model's training data to logs/NAME, which can then be read by TensorBoard.

Finally, we can add this callback to our model by adding it to the .fit method, like:

model.fit(X, y,
          batch_size=32,
          epochs=3,
          validation_split=0.3,
          callbacks=[tensorboard])

Notice that callbacks is a list. You can pass other callbacks into this list as well. Our model isn't defined yet, so let's put this all together now:

import tensorflow as tf
from tensorflow.keras.datasets import cifar10
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten
from tensorflow.keras.layers import Conv2D, MaxPooling2D
# more info on callbakcs: https://keras.io/callbacks/ model saver is cool too.
from tensorflow.keras.callbacks import TensorBoard
import pickle
import time

NAME = "Cats-vs-dogs-CNN"

pickle_in = open("X.pickle","rb")
X = pickle.load(pickle_in)

pickle_in = open("y.pickle","rb")
y = pickle.load(pickle_in)

X = X/255.0

model = Sequential()

model.add(Conv2D(256, (3, 3), input_shape=X.shape[1:]))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(256, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Flatten())  # this converts our 3D feature maps to 1D feature vectors
model.add(Dense(64))

model.add(Dense(1))
model.add(Activation('sigmoid'))

tensorboard = TensorBoard(log_dir="logs/{}".format(NAME))

model.compile(loss='binary_crossentropy',
              optimizer='adam',
              metrics=['accuracy'],
              )

model.fit(X, y,
          batch_size=32,
          epochs=3,
          validation_split=0.3,
          callbacks=[tensorboard])

Train on 17441 samples, validate on 7475 samples
Epoch 1/3
17441/17441 [==============================] - 13s 723us/step - loss: 0.6857 - acc: 0.5755 - val_loss: 0.6480 - val_acc: 0.6300
Epoch 2/3
17441/17441 [==============================] - 12s 712us/step - loss: 0.6488 - acc: 0.6299 - val_loss: 0.6327 - val_acc: 0.6527
Epoch 3/3
17441/17441 [==============================] - 12s 714us/step - loss: 0.6073 - acc: 0.6747 - val_loss: 0.5888 - val_acc: 0.6943

<tensorflow.python.keras.callbacks.History at 0x28bb5da42b0>

After having run this, you should have a new directory called logs. We can visualize the initial results from this directory using tensorboard now. Open a console, change to your working directory, and type: tensorboard --logdir=logs/. You should see a notice like: TensorBoard 1.10.0 at http://H-PC:6006 (Press CTRL+C to quit) where "h-pc" probably is whatever your machine's name is. Open a browser and head to this address. You should see something like:

Now we can see how our model did over time. Let's change some things in our model. To begin, we never added an activation to our dense layer. Also, let's try a smaller model overall:

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten
from tensorflow.keras.layers import Conv2D, MaxPooling2D
# more info on callbakcs: https://keras.io/callbacks/ model saver is cool too.
from tensorflow.keras.callbacks import TensorBoard
import pickle
import time

NAME = "Cats-vs-dogs-64x2-CNN"

pickle_in = open("X.pickle","rb")
X = pickle.load(pickle_in)

pickle_in = open("y.pickle","rb")
y = pickle.load(pickle_in)

X = X/255.0

model = Sequential()

model.add(Conv2D(64, (3, 3), input_shape=X.shape[1:]))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Flatten())  # this converts our 3D feature maps to 1D feature vectors
model.add(Dense(64))
model.add(Activation('relu'))

model.add(Dense(1))
model.add(Activation('sigmoid'))

tensorboard = TensorBoard(log_dir="logs/{}".format(NAME))

model.compile(loss='binary_crossentropy',
              optimizer='adam',
              metrics=['accuracy'],
              )

model.fit(X, y,
          batch_size=32,
          epochs=10,
          validation_split=0.3,
          callbacks=[tensorboard])

Train on 17441 samples, validate on 7475 samples
Epoch 1/10
17441/17441 [==============================] - 5s 289us/step - loss: 0.6523 - acc: 0.5997 - val_loss: 0.8054 - val_acc: 0.5663
Epoch 2/10
17441/17441 [==============================] - 5s 270us/step - loss: 0.5374 - acc: 0.7320 - val_loss: 0.5038 - val_acc: 0.7516
Epoch 3/10
17441/17441 [==============================] - 5s 268us/step - loss: 0.4617 - acc: 0.7812 - val_loss: 0.4969 - val_acc: 0.7658
Epoch 4/10
17441/17441 [==============================] - 5s 270us/step - loss: 0.4065 - acc: 0.8163 - val_loss: 0.4603 - val_acc: 0.7821
Epoch 5/10
17441/17441 [==============================] - 5s 271us/step - loss: 0.3434 - acc: 0.8477 - val_loss: 0.5129 - val_acc: 0.7692
Epoch 6/10
17441/17441 [==============================] - 5s 269us/step - loss: 0.2696 - acc: 0.8841 - val_loss: 0.5502 - val_acc: 0.7766
Epoch 7/10
17441/17441 [==============================] - 5s 272us/step - loss: 0.1932 - acc: 0.9228 - val_loss: 0.5870 - val_acc: 0.7755
Epoch 8/10
17441/17441 [==============================] - 5s 270us/step - loss: 0.1284 - acc: 0.9527 - val_loss: 0.6952 - val_acc: 0.7789
Epoch 9/10
17441/17441 [==============================] - 5s 272us/step - loss: 0.0670 - acc: 0.9792 - val_loss: 0.8279 - val_acc: 0.7802
Epoch 10/10
17441/17441 [==============================] - 5s 270us/step - loss: 0.0444 - acc: 0.9869 - val_loss: 0.9392 - val_acc: 0.7783

<tensorflow.python.keras.callbacks.History at 0x28b882af240>

Among other things, I also changed the name to NAME = "Cats-vs-dogs-64x2-CNN". Don't forget to do this, or you'll append to your previous model's logs instead by accident and it wont look too good. Let's check TensorBoard now:

Looking better! Immediately, however, you might notice the shape of validation loss. Loss is the measure of error, and it clearly looks like, after our 4th epoch, things began to sour. Interestingly enough, our validation accuracy still continued to hold, but I imagine it would eventually begin to fall. It's much more likely that the first thing to suffer will indeed be your validation loss. This should alert you that you're almost certainly beginning to over-fit. The reason why this happens is the model is constantly trying to decrease your in-sample loss. At some point, rather than learning general things about the actual data, the model instead begins to just memorize input data. If you let this continue, yes your "accuracy" in-sample will rise, but your out of sample, and any new data you attempt to feed the model, will perform poorly.

In the next tutorial, we'll talk about how you can use TensorBoard to optimize the model you use with your data.