Fine-tuning a custom Text analytics for Health model

Jais 0

I have created a custom TAH model. But the accuracy of the model is low for some EntityTypes. So I am planning to add more labelled entities for that EntityType and fine-tune the custom model which I have already created. But I could not find any topics related to this in the documentation. Is it possible to fine-tune or retrain a custom TAH model which I have already created? Any help would be greatly appreciated, Thank you....

santoshkc 4,520 Reputation points Microsoft Vendor

2024-02-12T05:07:03.4566667+00:00

Hi @Jais,

Thank you for reaching out to Microsoft Q&A forum!

It's important to carefully consider scenarios that use an automated feedback loop in fine-tuning a custom Text Analytics for Health model.

Additionally, it's crucial to test and evaluate the model before deploying it to a production environment to avoid model quality. Custom model training is an iterative process that is very sensitive to the input training data, so it's essential to ensure that the model is trained with high-quality data and that the training process is monitored and adjusted as needed. Train your custom Text Analytics for health model.

I hope you understand! Thank you.
Jais 0 Reputation points

2024-02-12T05:20:40.84+00:00

Hi @santoshkc Thanks for the quick response. The link you provided directs to the model training for custom TAH. I have a scenario where I have trained a custom model for various EntityTypes. But while in production, I have found that for some EntityTypes the accuracy degrades due to new data which the model has not seen. So I have collected and created a dataset to address this problem and further fine-tune the custom model V1 which I have already created. So it possible to further fine-tune a custom TAH model? In the language studio, I could only see options to create a new TAH model from the base TAH or overwrite an existing model, so is it not possible to partially tune the model with only the new data?
santoshkc 4,520 Reputation points Microsoft Vendor

2024-02-12T09:47:39.09+00:00

Hi @Jais,

Thank you for providing more details about your scenario. I apologize for the confusion with the link I provided earlier, just for an info provided the link for model training. As per your query you want to fine-tune a custom Text Analytics for Health model that you have already created. You can use the same process that you used to create the initial custom model, but this time you can use the new dataset that you have collected to fine-tune the model for the EntityTypes that have low accuracy.

To fine-tune your custom model, add the labelled entities to your dataset, train the model, and evaluate the model's performance. You can repeat this process as many times as needed to fine-tune the model for the EntityTypes that have low accuracy.

I hope this information helps.
Jais 0 Reputation points

2024-02-12T10:02:53.88+00:00

Hi @santoshkc Thank you for the response. I am currently using the Language studio for training a custom TAH model. While starting a new training job, I could only find 2 options, that is to create a new model or overwrite an existing model. I could not find any configurations for fine-tuning an existing model. So how can I fine-tune a model with the new data like you mentioned?
santoshkc 4,520 Reputation points Microsoft Vendor

2024-02-12T11:07:51.1633333+00:00

Hi @Jais, Sorry for the trouble you are facing. I request you to raise a support case through Azure portal. Thank you!