Azure AI Document Intelligence
An Azure service that turns documents into usable data. Previously known as Azure Form Recognizer.
1,387 questions
Azure AI studio Evaluation
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(54.400000000000006, 74%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA4%3C/text%3E%3C/svg%3E)
AkashNamdeo-4100
0
Reputation points
What could be the reasons behind Azure AI Studio returning NaN values for score metrics (retrieval, readiness, goodness) and failing to provide correct answers when evaluating test data, despite obtaining correct results when querying questions with data in the playground?
- For Query where I am getting relevant response
- For query where I am getting relevant answer in playground but not on test data during Evaluation.
{count} votes
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(214.4, 74%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAM%3C/text%3E%3C/svg%3E)
AshokPeddakotla-MSFT 27,491 Reputation points2024-04-02T10:26:41.4533333+00:00 Akash Namdeo Greetings and Welcome to Microsoft Q&A forum!
Can you add more details regarding the issue? Which documentation are you following?
There could be several reasons why Azure AI Studio is returning NaN values for score metrics and failing to provide correct answers when evaluating test data.
One possible reason is that the model may not have been trained on a diverse enough set of data, leading to overfitting.
Did you see Evaluation and monitoring metrics for generative AI already?
-
AshokPeddakotla-MSFT 27,491 Reputation points
2024-04-15T15:14:37.6866667+00:00 Akash Namdeo Did you get a chance to see my response?
Do let me know if you need further help.
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(233.6, 98%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAN%3C/text%3E%3C/svg%3E)
Akash Namdeo (A) 0 Reputation points2024-04-16T04:58:30.59+00:00 Sorry for the delay in response. Yes I have gone through the evaluation and monitoring metrics. I am following this documentation. If the model hadn't been trained on a sufficient dataset, why do I receive correct responses in the playground? It's only during evaluation that the same query doesn't yield accurate results.
-
AshokPeddakotla-MSFT 27,491 Reputation points
2024-04-23T13:25:27.1966667+00:00 Apologies for the delayed response. Unfortunately, this issue needs further investigation on what went wrong.
For a deeper investigation and immediate assistance on this issue, please file a support request @ https://aka.ms/azsupt?
Sign in to comment