How to copy a collection from one instance to another instance with Qdrant?

Question

In the process that i'm running, i need very low latency to be able to process a job, so i use a local instance of qdrant db to be able to insert everything very fast to it.

After finishing the job, i want to copy the whole collection from the local instance to a cloud instance of Qdrant.

I was just wondering if there is any better way of doing this other than simply scrolling through the whole collection and inserting it to the other instance.

Answer 1

Since there isn't any other point of contact as such between a local(in-memory or persisted) and a server instance of Qdrant, scrolling through the local collection to upload the points should be the appropriate course of action.

Answer 2

i was able to solve this particular problem using the client.migrate method, on this way:

cloud_client = QdrantClient(
            url = qdrant_url,
            api_key = key)

cloud_client.recreate_collection(
        collection_name=collection_name,
        vectors_config=VectorParams(size=2048, distance=Distance.COSINE,on_disk = True)
    )

local_client = QdrantClient("localhost",port=6333)

local_client.migrate(cloud_client,[collection_name],batch_size = 100,recreate_on_collision=True)

with larger batches it has failed. Also, looking throug the code, it seems to be implementing just a scroll over every item, as @Anush suggested:

https://github.com/qdrant/qdrant-client/blob/6d019e67a133bea87a96bce388f5b901cdae1287/qdrant_client/migrate/migrate.py#L104