KDB/Q How to implement moving rank efficiently?

338 Views Asked by At

I am trying to implement a moving rank function, taking parameters of n, the number of items, and m, the column name. Here is how I implement it:

mwindow: k){[y;x]$[y>0;x@(!#x)+\:!y;x@(!#x)+\:(!-y)+y+1]};

mrank: {[n;x] sum each x > prev mwindow[neg n;x]};

But this seems to take quite some time if n is moderately large, say 100.

I figure it is because it has to calculate from scratch, unlike msum, which keeps a running variable and only calculate the difference between the newly added and the dropped.

1

There are 1 best solutions below

1
terrylynch On

There's a number of general sliding window functions here that you can use to generate rolling lists on which to apply your rank: https://code.kx.com/q/kb/programming-idioms/#how-do-i-apply-a-function-to-a-sequence-sliding-window

Those approaches seem to fill the lists out with zeros/nulls however which I think won't really suit your use of rank. Here's another possible approach which might be more suitable to rank (though I haven't tested this for performance on the large scale):

q)mwin:{x each (),/:{neg[x]sublist y,z}[y]\[z]}
q)update r:mwin[rank;4;c] from ([]c:10?100)
c  r
----------
84 ,0
25 1 0
31 2 0 1
0  3 1 2 0
51 1 2 0 3
29 2 0 3 1
25 0 3 2 1
73 2 1 0 3
0  2 1 3 0
6  2 3 0 1
q)update r:last each mwin[rank;4;c] from ([]c:10?100)
c  r
----
38 0
72 1
13 0
77 3
64 1
9  0
37 1
79 3
97 3
63 1
q)