Azure AI Chatbot: need to ask the same question thrice before I get an answer

Question

• Deployment: chatgpt-4
• Api_Version: 2023-12-01

Settings:

	"dataSources": [
		{
			"type": "AzureMLIndex",
			"parameters": {
				"version": "1",
				"queryType": "vectorSimpleHybrid",
				"inScope": true,
				"strictness": 1,
				"topNDocuments": 20
			}
		}
	],
	"temperature": 0.8,
	"top_p": 0,
	"max_tokens": 2400,
	"stop": null,
	"stream": true

I have a container with documents that are used to answer the questions.

However, I don't get the right answer untilI ask it three times. The first two times it just tells me the it can't find the answer in the documents. Then the rhird time it usually gets it right.

Even if I phrase it differently or in three different ways. I need to ask the question three times in order to get my answers.

It seems like it needs three times to get to the right source document. Even though I don't use that many source documents, muy total is 4. But 3 of them mention something about this topic, so that might still be an issue.

Answer 1

Hello @Dirk Broenink

Thanks for posting in the Microsoft Q&A!

Your Azure AI Chatbot requires multiple attempts to provide the correct answer to your question. Despite your efforts to phrase the question differently or in multiple ways, the initial responses indicate that the answer couldn't be found in the documents, but eventually, on the third attempt, the correct answer is provided. This behavior might stem from various factors, including the complexity of the query, the settings of your AI model, or the coverage of your document sources. Despite having only four source documents, if three of them mention the topic in question, it's possible that the AI model needs multiple attempts to locate the relevant information accurately. To address this issue, consider adjusting the parameters of your Azure AI model, such as the temperature or max_tokens settings, to fine-tune its response generation. Additionally, review the coverage of your document sources to ensure comprehensive information retrieval. By optimizing these factors, you may improve the accuracy and efficiency of your Azure AI Chatbot's responses, reducing the need for repeated queries to obtain the correct answer.

If you found this solution useful, consider accepting it.