Get the token usage from semantic kernel C#

Question

I am trying out the semantic kernel in C# for a project. I am using the ChatCompletionService with Azure OpenAI and a few simple plugins that I created. I am not able to figure out how to get the token usage for a query. Any help would be appreciated. For example, in this solution that microsoft has, is there a way to get the token usage after each question (GetStreamingChatMessageContentsAsync)? One link I found was about logging in open telmetry standard. But it does not log the token usage. Any help is appreciated.

Answer 1

Considering the solution link you referenced in your question, one possible way to get this information is by making use of Metadata property of FunctionResult. Token information is contained in Usage key in there which is of type CompletionsUsage.

Your code would be something like:

var result = await kernel.InvokeAsync(function-to-invoke)
var metadata = result.Metadata
if (metadata.ContainsKey("Usage"))
{
    var usage = (CompletionsUsage)metadata["Usage"];
    Console.WriteLine($"Token usage. Input tokens: {usage.PromptTokens}; Output tokens: {usage.CompletionTokens}");
}

Please note that this information is not available just yet for streaming responses. For more details, please see this issue on GitHub: https://github.com/microsoft/semantic-kernel/issues/4691.