Loading a Huggingface Model with Microsofts Semantic Kernel in C# / VB.NET

Question

Microsoft presented it's new library Semantic Kernel to code own chat programs like ChatGPT with .NET.

In their documentation they are telling, that you can either use OpenAI´s LLM, Azure LLM or Huggingfaces LLM.

I prefer using Huggingfaces LLM, because I prefer running local LLM for free, instead of paying for a cloud service.

But I don't see any documentation at the Microsoft website, a Youtube video, somebody at stackoverflow or anywhere else at the internet who managed to load a local LLM with Semantic Kernel with C#/VB.NET.

So my question is, does anybody know´s how to load and use them?

Thanks!

Answer 1

I've been learning about this, and I see that the apiKey parameter is provided again, but HuggingFace doesn't seem to provide an online service like OpenAi. I understand that you need to start your own service https://github.com/microsoft/semantic-kernel/tree/3451a4ebbc9db0d049f48804c12791c681a326cb/samples/apps/hugging-face-http-server , withHuggingFaceTextCompletionService, specify your own Endpoint.

Answer 2

For open source models, you can disregard API keys. I’ve used a couple Llama 2 models. API keys, as far as I’ve seen are only required by OpenAI models.

You can download the model to your computer to any drive/folder you like. In your app, using the API, you specify the path as a string.