I am using the SciPy implementation of KL-divergence ([http://docs.scipy.org/doc/scipy-dev/reference/generated/scipy.stats.entropy.html]) for two different numpy arrays.
The first one , lets say "base_freq" has a standard length of 2000 The second one, "test_freq" length can take different values depending on the sample. So let's say that its length is 8000.
How can I compute the KL divergence when these two do not have the same length???
My thought was to break down the second array ("test_freq") to a number of arrays with 2000 length. But how is this done?? And what happens when the "test_freq" gets a sample of 250 length ?
I should preface by saying that I'm no information theory expert. For the one application in which I used KL-divergence, I was comparing two images pixel-wise to compute the number of bits lost. If the images had different sizes, your proposed approach would require that for each pixel in the smaller image I choose the corresponding pixel in the larger--not any old pixel. My understanding was that KL-divergence only makes sense if you're comparing two signals sampled the same way (i.e. the same temporal or spatial sampling interval).
If you want to do what you propose, you may use
numpy.random.choice
: