I have been playing with some publicly available Datasets on RNNs to get used to it and learn tensorflow. Now I wish to use my own dataset, which are basically exponential signals with various time constants.
# I want to perform a classification task based on the steepness of the exponentials. This property is the 'Time constant- Tau'.
# I have created 2 different bins of tau values in python and one-hot encoded them to use as labels.
# I have created a python generator for generating the signals and labels.
Q 1. Upto this point, it's all good, but what I don't understand is how can format this input for the RNN? In MNIST, we make chunks of 28x28 because pixels are the features. But for my signals what will be the chunks?
Q 2. MNIST dataset already comes formatted and all I did was select batch numbers. Also there is "mnist.train.next_batch" to pass according to the batch size. In terms of my data, I am not quite sure how to make the equivalent of "mnist.train.next_batch".
You must be logged in to post. Please login or register an account.
In the RNN, you decide the chunk size. We did 28 chunks of 28 pixels each, but you can split up your data however you see fit (obviously it needs to be the same size at the end of the day, but you can do whatever you like. )
If you check out: https://pythonprogramming.net/train-test-tensorflow-deep-learning-tutorial/, you can see how I personally handle the batches with a custom dataset. A snippet from that tutorial:
for epoch in range(hm_epochs): epoch_loss = 0 i=0 while i < len(train_x): start = i end = i+batch_size batch_x = np.array(train_x[start:end]) batch_y = np.array(train_y[start:end])
_, c = sess.run([optimizer, cost], feed_dict={x: batch_x, y: batch_y}) epoch_loss += c i+=batch_size
print('Epoch', epoch+1, 'completed out of',hm_epochs,'loss:',epoch_loss)
We use a while loop, and wait til we've gone over all of the sample data. i starts at 0, and ends at i+batch_size. After each loop in this while loop, we add the batch_size to i, effectively creating batches.
Good luck!
-Harrison 8 years ago
You must be logged in to post. Please login or register an account.
Thanks a lot for your help Harrison. I understood the while loop you have used for creating batches, however, with my data I am finding it difficult to recreate the batch_x and batch_y.
Q. I want X to be the train inputs and Y to be the labels. What is the best way to create them into batches? Q. I tried to do """"batch_x = np.array(X[start:end])"""" , directly using X which gave me index error.
I will give an example of the function I created for generating the data-