How do I backpropagate gradients onto the input of time step 1 in an LSTM network in Tensorflow?

Question

In Tensorflow, I feed a matrix X of input vectors into an LSTM network. The first vector of the input matrix (time step 1) is a one-hot encoded vector, whereas the following vectors of X (following time steps) are zero vectors. Now, I want to backpropagate the gradients of the loss onto the input vector at time step 1.

If I use the following operation, I get the gradients of the input, but for every time step.

gradients_input = sess.run(tf.gradients(loss, X), feed_dict={X:batch_x, Y:batch_y}) 

Any idea how I can compute the gradients only for time step 1?