Improve loading of json if end of json is not found for metrics

[Yleisnero]

Some LLMs struggle to generate a complete and correct JSON (e.g. mistral-7b-instruct). Also see issue #743.
However, often, only the closing bracket of the JSON is missing. This can be fixed easily by adding the bracket at the end of the input string.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
evals-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jun 6, 2024 8:47am

[FaizaQamar]

Hey, I couldn't understand your solution. The same lines of code are in the utils.py file related to missing paranthesis. But the problem persists. Can you please explain your solution? @Yleisnero

[Yleisnero]

@FaizaQamar I encountered the problem that Mistral was generating a correct json but was leaving out the closing curly bracket. So this solution only fixes this specific case.

[MINJIK01]

I attached bracket at the end of the string, but it still doesn't work and reply same error.

I wrote like below.

test_case = LLMTestCase(
input="Write me a joke}",
actual_output="Why did the chicken go to the doctor? Because it had a fowl cough! I hope that made you cluck with laughter!",
)

[Yleisnero]

This is not a fix that the user should apply. As @FaizaQamar mentioned, this code is in the utils.py because it is a merged pull request. This fixes some problems, but not everything. LLMs are not deterministic and unreliable. If you still encounter the issue, this here is not the solution.