Adding filters to ConversationalRetrievalChain [langchain/pinecone]

Question

I'm trying to add filters/metadata to retrieve documents from pinecone in a specified timeframe.

I have added the following filter to the retriver:

retriever = vector_store.as_retriever(
    search_type="similarity_score_threshold",
    search_kwargs={
        "k": RETRIEVER_K,
        "score_threshold": RETRIEVER_THRESHOLD,
        "filter": {"publisheddate": {"$gte": "2023-01-01T00:00:00.000Z"}},
    },
)

This doesn't seem to work.

I'm using this retriver in ConversationalRetrievalChain

chain = ConversationalRetrievalChain(
    combine_docs_chain=combine_docs_chain,
    retriever=retriever,
    question_generator=question_generator_chain,
    return_source_documents=True,
    verbose=True,
)

Pinecone index has a value named publisheddate, see below:

How can I retrieve docs from pinecone vector db with publisheddate in a specific time frame?