Cassandra paging on a primary key list does not work

Question

I have a problem with the pagination in cassandra with python connector. I would like to paginate on a list of primary keys.

Here is the schema of the table:

client_data.store (
    id text PRIMARY KEY,
    "additionalCols" text,
    format text,
    name text
)

Here is what I have in database:

database content

I would like to have pages of 5 rows by filtering on id from 1 to 15.

I make a first query :

stores = (
            Store.Cql.objects.all()
            .filter(id__in=[str(x) for x in range(1, 15)])
            .limit(5)
        )

and I get the columns with id :

['1', '10', '11', '12', '13']

I take the last element, in this case '13' then I do

last_element_pk = '13'
token = Token(last_element_pk)
stores = (
            Store.Cql.objects.all()
            .filter(id__in=[str(x) for x in range(1, 15)], pk__token__gt=(token))
            .limit(5)
        )

and I get :

['1', '10', '11', '12', '14']

I don't understand why I have common ids?!

I use cassandra:4.0.1, and cassandra-driver = "^3.25.0"

Thank you for your help

Answer 1

Simple reason is that token is hash function which calculates a hash value from the partition key. So saying give me key with formula gt>13 and saying give me tokens with formula gt>token(13) is different thing. You cannot order partition keys as they are never stored in serial order. So what you are getting is correct as per the query issued. You are asking give keys whose token value is greater than token value of 13 and give 5 such keys. So the response is correct.

Partition keys cannot be ordered by normal partition keys of type(int,long etc)