.Net: New Feature: Support Vertex AI Fine-tuned Models

[nguyenlamlll]

---

name: Support Vertex AI Fine-tuned Models
about: The current GeminiChatCompletionClient implementation only supports publisher models (like gemini-x-flash or gemini-x-pro, etc.). However, if we want to use fine-tuned models from Vertex AI, we must use a different URI format. It seems like we do not support fine-tuned models at the moment.

---

The problem

The current GeminiChatCompletionClient implementation only supports publisher models using this format

this._chatGenerationEndpoint = new Uri($"https://{location}-aiplatform.googleapis.com/{versionSubLink}/projects/{projectId}/locations/{location}/publishers/google/models/{this._modelId}:generateContent");
this._chatStreamingEndpoint = new Uri($"https://{location}-aiplatform.googleapis.com/{versionSubLink}/projects/{projectId}/locations/{location}/publishers/google/models/{this._modelId}:streamGenerateContent?alt=sse");

However, according to the Vertex AI REST API doc, fine-tuned models must be accessed using the format: projects/{project}/locations/{location}/endpoints/{endpoint}

Proposed solution

Perhaps we can add a new constructor parameter to distinguish between publisher models and fine-tuned models.
Using that boolean param, we can construct the endpoint format accordingly.

public GeminiChatCompletionClient(
    HttpClient httpClient,
    string modelId,
    Func<ValueTask<string>> bearerTokenProvider,
    string location,
    string projectId,
    VertexAIVersion apiVersion,
    bool isFineTunedModel = false, // New parameter
    ILogger? logger = null)
    : base(
        httpClient: httpClient,
        logger: logger,
        bearerTokenProvider: bearerTokenProvider)
{
    // ...
    if (isFineTunedModel)
    {
        // Fine-tuned model endpoint format
        this._chatGenerationEndpoint = new Uri($"https://{location}-aiplatform.googleapis.com/{versionSubLink}/projects/{projectId}/locations/{location}/endpoints/{this._modelId}:generateContent");
        this._chatStreamingEndpoint = new Uri($"https://{location}-aiplatform.googleapis.com/{versionSubLink}/projects/{projectId}/locations/{location}/endpoints/{this._modelId}:streamGenerateContent?alt=sse");
    }
    else
    {
        // Publisher model endpoint format (existing behavior)
        this._chatGenerationEndpoint = new Uri($"https://{location}-aiplatform.googleapis.com/{versionSubLink}/projects/{projectId}/locations/{location}/publishers/google/models/{this._modelId}:generateContent");
        this._chatStreamingEndpoint = new Uri($"https://{location}-aiplatform.googleapis.com/{versionSubLink}/projects/{projectId}/locations/{location}/publishers/google/models/{this._modelId}:streamGenerateContent?alt=sse");
    }
}

I do not have a deep understanding of the project architecture, so my suggestion may be naive. Please feel free to propose something else.

Related

• I have provided a small console project here, explaining that the current version 1.62 does not work with fine-tuned models: How to use fine-tuned models on Vertex AI?
• Some related issues and discussions: .Net: Integrating with VertexAI Endpoint #10889 and Discussion #10866

[nguyenlamlll]

I'm creating a small PR for this one.