I get very different comparison results for the same texts. [sentence-transformers/all-mpnet-base-v2]

13 Views Asked by At

I submit the same text, but the similarity value is different. It feels like the model chooses one of 3-4 behavior patterns. How can I make the results deterministic?

    if find(IMAGE_CLASS).get_attribute('class') == "keysMaker_picdiv":
        description = find(IMAGE_TITLE).get_attribute('title')
        description_value = MODEL.encode(description, convert_to_tensor=True)

        similarity_value = util.pytorch_cos_sim(input_value, description_value)[0][0]
        print(f'{index}. {description}\nСхожесть {similarity_value:.3f}. {"Проходит!" if similarity_value >= N_FACTOR else "Нет!"}')
        if similarity_value >= N_FACTOR:
            return True

Here I trying to make it deterministic on my own. Not showing results.

def get_model(SEED):
    random.seed(SEED)
    numpy.random.seed(SEED)
    torch.manual_seed(SEED)
    torch.cuda.manual_seed(SEED)
    torch.cuda.manual_seed_all(SEED)
    torch.backends.cudnn.deterministic = True
    torch.backends.cudnn.benchmark = False
    MODEL = SentenceTransformer("all-mpnet-base-v2")
    print(checksum(MODEL))
    MODEL = MODEL.to('cpu')
    return MODEL
0

There are 0 best solutions below