so I wrote the following code:
- I have the number of tokens.
- I looked up the price:
gpt-4 $30.00 / 1M tokens
def print_embedding_cost(texts):
import tiktoken
enc = tiktoken.encoding_for_model('gpt-4')
total_tokens = sum([len(enc.encode(page.page_content)) for page in texts])
print(f'Total Tokens: {total_tokens}')
print(f'Embedding Cost in USD: { (30 / 1_000_000) * total_tokens}')
print_embedding_cost(chunks)
Thinking: 30 USD for 1 Million tokens = price per token. Somehow this results into a very high price, which isn't correct.