Training with an instance segmentation dataset

[DP1701]

Search before asking

• I have searched the Multimodal Maestro issues and found no similar feature requests.

Question

Hi all, does anyone know a website or YouTube tutorial that explains how to fine tune Florence-2 or Paligemma_2 with an instance segmentation dataset? Is this also possible with maestro?

Additional

No response

[SkalskiP]

Hi @DP1701 👋🏻 let me convert this question into a GH discussion first. BTW I have a question: I'm thinking about launching a Discord server dedicated to VLM fine-tuning with Maestro) and while talking about current isses I'm trying to understand if people would like such a server to be created.

[SkalskiP]

In the past I've experimented with PaliGemma and PaliGemma 2 fine tuning for instance segmentation. I hope you find this helpful.

[DP1701]

Hi @SkalskiP
Thanks for the information. I will have a look at it. Do you think it can also be used for Florence-2 in a similar way?

Regarding your question about whether a Discord server is worthwhile: definitely. Personally, I am looking for opportunities to talk to others about this topic, because I have discovered VLMs for myself and find it interesting.