Optimizing Data Insertion Efficiency in Milvus Collections: Addressing Delays with Minimal Data Sets

19 Views Asked by At

When inserting minimal data into the Milvus collection, a significant amount of time is consumed, and the reason for this delay is not apparent. Could you kindly provide recommendations for resolving this issue?

my collection details:

fields = [FieldSchema(name='id', dtype=DataType.INT64, description='ids', max_length=200, is_primary=True, auto_id=False),
FieldSchema(name='human_id', dtype=DataType.INT64, description='human_ids', max_length=2000, is_primary=False, auto_id=False),
FieldSchema(name='human_name', dtype=DataType.VARCHAR, description='names of human classes', max_length=1000, auto_id=False),
            FieldSchema(name='vector', dtype=DataType.FLOAT_VECTOR, description='embedding vectors', dim=embedding_dimensions),]

schema = CollectionSchema(fields=fields, description='classification')
collection = Collection(name=collection_name, schema=schema)
index_params = {
 'metric_type': metric_type,
 'index_type': index_type,
 'params':  {"nlist": parameter_number}}
collection.create_index(field_name="vector", index_params=index_params)

METRIC_TYPE = "COSINE"
INDEX_TYPE = "IVF_FLAT"
PARAMETER_NUMBER = 128

collection = Collection(collection_name)
collection.insert(data)`

Adjusting Shared Number Multiplicity for Improved Loading Efficiency in Collection Management

0

There are 0 best solutions below