question

JussiMuilu-8813 avatar image
0 Votes"
JussiMuilu-8813 asked YutongTie-MSFT commented

How to use customized model with Speech Translate?


We are trying to create speech translation app using customized models for speech recgonition and translation. We have created a custom speech recognition model that performs quite well on our domain. We have also trained some custom translation models for the translation.

Previously we have used the results of speech recognition as input to translation service but in order to simplify the app and remove some latency we are looking for using the SpeechTranslator of the SpeechSDK


Now the problem is that I can't get the speechSDK TranslationRecognizer (javascript) to use my custom speech recognition model.

When using regular speech recognition I create a SpeechRecgonizer config using auth token and then set the custom enpointId value.

I tried the same with the TranslationRecognizer. The translation works but it clearly uses the baseline speech recgonition model instead of our custom model. I tried to create the TranslationRecognizer using the configuration created with fromEndpoint methdo but didn't get any results from there...


The custom speech recognition model is using Finnish -language

azure-cognitive-servicesazure-speechazure-translator
· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi there,

Is there any error returned when you ran your model?


Regards,
Yutong

0 Votes 0 ·

No, I dont get any errors. It seems to be working but the Speech Recognition results indicate that a baseline model is being used instead of customized model.

This is how I set up the TranslationRecognizer.

 var audioConfig;
 audioConfig = SpeechSDK.AudioConfig.fromDefaultMicrophoneInput();
    
 var speechConfig;
 speechConfig = SpeechSDK.SpeechTranslationConfig.fromAuthorizationToken(<AuhtToken>, <AzureRegion> );
            
 speechConfig.endpointId = <MyEndpointId>;
            
 speechConfig.speechRecognitionLanguage =  <SourceLanguageCode>;         
 speechConfig.addTargetLanguage(<TargetLanguageCode);
    
            
 Translator = new SpeechSDK.TranslationRecognizer(speechConfig, audioConfig);






0 Votes 0 ·

1 Answer

YutongTie-MSFT avatar image
0 Votes"
YutongTie-MSFT answered YutongTie-MSFT commented

Hello,

Thanks for your response. Please try below sample code for how to use your custom model:
var sourceLanguageConfig = SourceLanguageConfig.FromLanguage("de-DE", "The Endpoint ID for your custom model.");
var recognizer = new SpeechRecognizer(speechConfig, sourceLanguageConfig, audioConfig);

SpeechRecognitionLanguage and EndpointId set methods are deprecated from the SpeechConfig class in C#. The use of these methods are discouraged, and shouldn't be used when constructing a SpeechRecognizer. I think this should be the error in your sample


More details please check :
https://docs.microsoft.com/en-us/azure/cognitive-services/speech-service/how-to-specify-source-language?pivots=programming-language-csharp

Sorry for the delay and hope this helps.

Regards,
Yutong

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

anonymous user FYI please check on this.

0 Votes 0 ·