.Net: Thread handling issues with Response Agent streaming response

[thomasjlittle]

I am using the OpenAIResponseAgent in .Net and am trying to use a thread to handle conversation history. I am unable to pass in a OpenAIResponseAgentThread to the Agent's InvokeStreamingAsync call but am able to use a ChatHistoryAgentThread (there is no issue here I just want to check that this is the intended behavior).

The issue I am seeing is that the thread only has the user messages after running a streaming response and does not contain any assistant or tool call messages, however, if I invoke a non-streaming response it will have the assistant and tool call messages as well as the user messages. Do I need to manually add the streamed responses to the agent thread when using InvokeStreamingAsync?

using System.ClientModel;
using ConsoleApp4;
using Microsoft.Extensions.Configuration;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Agents;
using Microsoft.SemanticKernel.Agents.OpenAI;
using Microsoft.SemanticKernel.ChatCompletion;
using OpenAI.Responses;

#pragma warning disable SKEXP0110
#pragma warning disable OPENAI001

var configuration = new ConfigurationBuilder()
    .AddJsonFile("appsettings.json")
    .Build();

await AddPluginsAsync();
async Task AddPluginsAsync()
{
    // Set vars
    string openAiKey = configuration["openAIKey"] ?? throw new InvalidOperationException("OpenAI key is not set in appsettings.json");
    
    // Create the responses agent 
    OpenAIResponseClient client = new OpenAIResponseClient("gpt-4.1", new ApiKeyCredential(openAiKey));
    OpenAIResponseAgent responseAgent = new(client);
    
    // Create a plugin that defines the tools to be used by the agent.
    KernelPlugin plugin = KernelPluginFactory.CreateFromType<MenuPlugin>();
    responseAgent.Kernel.Plugins.Add(plugin);

    ICollection<ChatMessageContent> messages =
    [
        new ChatMessageContent(AuthorRole.User, "What is the special soup and its price?"),
        new ChatMessageContent(AuthorRole.User, "What is the special drink and its price?"),
    ];
    Console.WriteLine("Messages to be sent to the agent:");
    foreach (ChatMessageContent message in messages)
    {
        Console.WriteLine(message);
    }
    
    // Creat agent thread
    // OpenAIResponseAgentThread agentThread = new OpenAIResponseAgentThread(client); //<------------This will throw an error
    ChatHistoryAgentThread? agentThread = new ChatHistoryAgentThread();

    // Invoke the agent and output the response
    var responseItems = responseAgent.InvokeStreamingAsync(messages, agentThread);
    string result = "";
    await foreach (StreamingChatMessageContent content in responseItems)
    {
        result += content.Content;
    }
    Console.WriteLine("Response from the agent:");
    Console.WriteLine(result);
    
    // Print the thread messages
    Console.WriteLine("Messages in the agent thread:");
    var messageList  = await agentThread.GetMessagesAsync().ToListAsync();
    foreach (ChatMessageContent message in messageList)
    {
        Console.WriteLine(message.Role + ": " + message.Content);
    }
}

This responds with the following:

Messages to be sent to the agent:
What is the special soup and its price?
What is the special drink and its price?

Response from the agent:
The special soup is Clam Chowder, priced at $4.95.
The special drink is Chai Tea, priced at $2.95.

Messages in the agent thread:
user: What is the special soup and its price?
user: What is the special drink and its price?

Note that the tool calls and assistant messages are not present in the thread after streaming the response.

This example was run with the following packages:
Microsoft.SemanticKernel 1.60.0
Microsoft.SemanticKernel.Agents.OpenAI 1.60.0
Microsoft.SemanticKernel.Core 1.60.0

[crickman]

@markwallace-microsoft - Do you have insight into the dynamics here?

[markwallace-microsoft]

Hi @thomasjlittle
I have updated the conversation state samples here: #12777

Also to understand how Response manages threads take a look here: https://platform.openai.com/docs/guides/conversation-state?api-mode=responses for more information

[thomasjlittle]

Awesome thank you, I will take a look at these