How to use customized model with Speech Translate?

Question

question

JussiMuilu-8813 asked Mar 11, '21 YutongTie-MSFT commented Mar 23, '21

How to use customized model with Speech Translate?

We are trying to create speech translation app using customized models for speech recgonition and translation. We have created a custom speech recognition model that performs quite well on our domain. We have also trained some custom translation models for the translation.

Previously we have used the results of speech recognition as input to translation service but in order to simplify the app and remove some latency we are looking for using the SpeechTranslator of the SpeechSDK

Now the problem is that I can't get the speechSDK TranslationRecognizer (javascript) to use my custom speech recognition model.

When using regular speech recognition I create a SpeechRecgonizer config using auth token and then set the custom enpointId value.

I tried the same with the TranslationRecognizer. The translation works but it clearly uses the baseline speech recgonition model instead of our custom model. I tried to create the TranslationRecognizer using the configuration created with fromEndpoint methdo but didn't get any results from there...

The custom speech recognition model is using Finnish -language

azure-cognitive-services azure-speech azure-translator

· 2

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

YutongTie-MSFT · Mar 12, 2021 at 12:38 AM

Hi there,

Is there any error returned when you ran your model?

Regards,
Yutong

0 ·

JussiMuilu-8813 YutongTie-MSFT · Mar 16, 2021 at 10:56 AM

No, I dont get any errors. It seems to be working but the Speech Recognition results indicate that a baseline model is being used instead of customized model.

This is how I set up the TranslationRecognizer.

 var audioConfig;
 audioConfig = SpeechSDK.AudioConfig.fromDefaultMicrophoneInput();
    
 var speechConfig;
 speechConfig = SpeechSDK.SpeechTranslationConfig.fromAuthorizationToken(<AuhtToken>, <AzureRegion> );
            
 speechConfig.endpointId = <MyEndpointId>;
            
 speechConfig.speechRecognitionLanguage =  <SourceLanguageCode>;         
 speechConfig.addTargetLanguage(<TargetLanguageCode);
    
            
 Translator = new SpeechSDK.TranslationRecognizer(speechConfig, audioConfig);

0 ·

Answer 1 · 2021-03-23T09:13:19.477Z

YutongTie-MSFT answered Mar 23, '21 YutongTie-MSFT commented Mar 23, '21

Hello,

Thanks for your response. Please try below sample code for how to use your custom model:
var sourceLanguageConfig = SourceLanguageConfig.FromLanguage("de-DE", "The Endpoint ID for your custom model.");
var recognizer = new SpeechRecognizer(speechConfig, sourceLanguageConfig, audioConfig);

SpeechRecognitionLanguage and EndpointId set methods are deprecated from the SpeechConfig class in C#. The use of these methods are discouraged, and shouldn't be used when constructing a SpeechRecognizer. I think this should be the error in your sample

More details please check :
https://docs.microsoft.com/en-us/azure/cognitive-services/speech-service/how-to-specify-source-language?pivots=programming-language-csharp

Sorry for the delay and hope this helps.

Regards,
Yutong

· 1

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

YutongTie-MSFT · Mar 23, 2021 at 09:14 AM

anonymous user FYI please check on this.

0 ·

question