Voice Recognition - handling no sound/rumour

FumetsujoFumetsujo USMember
edited June 2016 in Xamarin.Android

Good evening,

I got an issue where I need to handle a "not understood vocal response" (speech to text from Google).
Now I get the popup "try again" when the user says something unintelligibly, but I want to handle it instead (restart the audio recording automatically).

The idea is that the user can continuous speak with the a.i. (think about Jarvis or Amazon's echo).

Things I tried: under the if (resultVal == Result.Ok) setting an else statement, adding extra options in the voiceIntent.

Note:
The speech to text solution should be compatible with android 4.03 till 6 (present).

My code:
_ private void RecordVoice(object s, EventArgs e) //<-- might change it to a void without params

    {
        try { 
        var result = Question;
        result.Text = string.Empty;
        var voiceIntent = new Intent(RecognizerIntent.ActionRecognizeSpeech);
        voiceIntent.PutExtra(RecognizerIntent.ExtraLanguageModel, RecognizerIntent.LanguageModelFreeForm);
        voiceIntent.PutExtra(RecognizerIntent.ExtraPrompt, "Spreek :)");
        voiceIntent.PutExtra(RecognizerIntent.ExtraSpeechInputCompleteSilenceLengthMillis, 1500);
        voiceIntent.PutExtra(RecognizerIntent.ExtraSpeechInputPossiblyCompleteSilenceLengthMillis, 1500);
        voiceIntent.PutExtra(RecognizerIntent.ExtraSpeechInputMinimumLengthMillis, 15000);
        voiceIntent.PutExtra(RecognizerIntent.ExtraMaxResults, 1);
        voiceIntent.PutExtra(RecognizerIntent.ExtraLanguage, Java.Util.Locale.Default);
       // voiceIntent.PutExtra(RecognizerIntent.ExtraConfidenceScores, 0); // "heel flexibel om te zorgen dat 'retry' niet getoond wordt".
        StartActivityForResult(voiceIntent, VOICE);


        }
        catch
        {
            RunOnUiThread(() => Answer.Text = "Audio-apparaat niet toegankelijk."); 
        RunOnUiThread(() => spreken.Eir_Stem("Audio-apparaat niet toegankelijk."));

    }
}

    protected override void OnActivityResult(int requestCode, Result resultVal, Intent data)
    {
        if (requestCode == VOICE)
        {
            if (resultVal == Result.Ok)
            {
                var matches = data.GetStringArrayListExtra(RecognizerIntent.ExtraResults);
                var result = Question;
                result.Text = matches[0].ToString().ToLower();// belangrijk tolower()!
                Ask.PerformClick();
            }
        }
         base.OnActivityResult(requestCode, resultVal, data);
    }

_

Thank you for the help in advance.

Regards,
Jonathan N.

Posts

  • FumetsujoFumetsujo USMember
    edited June 2016

    Hi everybody,

    I believe that my initial question wasn't clear, my apologies for that.
    The good thing is that I've found a solution to the "longer wait before speech timeout" issue.
    Seems there isn't a built in function for the Google Speech Api to set the listening timeout.

    I've found a workaround though:
    use a timer (Timer timer) , a counter (int 'clock' here) and a and a bool (bool 'timoutopvangen' here).

    Example:

    _

        private async void OnTimedEvent(object sender, System.Timers.ElapsedEventArgs e)
        {
    
    
            clock++;
    
    
            switch (clock)
            {
                case 7:
                    if (timoutopvangen)
                    {
                        try
                        { 
                            timoutopvangen = false; // opvangen timeout
                            RunOnUiThread(() => RecordVoice());
    
                        }
                        catch
                        {
                            RunOnUiThread(() => Answer.Text = "Kon de Voice Recognition niet resetten. Gebruik je een verouderde of meest recente Android versie?");
                            RunOnUiThread(() => spreken.Eir_Stem("Kon de Voice Recognition niet resetten. Gebruik je een verouderde of meest recente Android versie?"));
    
                        }
    
    
    
                    }
    
                    break;
    

    private void RecordVoice()
    {

            clock = 0;
    
            if (toelatenTts)
            { try
                {
    
                  voiceIntent = new Intent(RecognizerIntent.ActionRecognizeSpeech);
                voiceIntent.PutExtra(RecognizerIntent.ExtraLanguageModel, RecognizerIntent.LanguageModelFreeForm);
                voiceIntent.PutExtra(RecognizerIntent.ExtraPrompt, "Spreek :)");
                voiceIntent.PutExtra(RecognizerIntent.ExtraSpeechInputCompleteSilenceLengthMillis, 1500);
                voiceIntent.PutExtra(RecognizerIntent.ExtraSpeechInputPossiblyCompleteSilenceLengthMillis, 1500);
                voiceIntent.PutExtra(RecognizerIntent.ExtraSpeechInputMinimumLengthMillis, 1500); //<!-- minimum tijd recording https://github.com/anubhavranjan/Speech-To-Text-In-Droid/blob/master/SpeechToText/MainActivity.cs
                voiceIntent.PutExtra(RecognizerIntent.ExtraMaxResults, 1);
                voiceIntent.PutExtra(RecognizerIntent.ExtraLanguage, Java.Util.Locale.Default);
                // voiceIntent.PutExtra(RecognizerIntent.ExtraConfidenceScores, 0);  
                StartActivityForResult(voiceIntent, VOICE);
                    timoutopvangen = true;
    
                }
                catch
            {
                    toelatenTts = false; // bug vermijden (herstarten om te kijken of audio-apparaat toch werkt).
    
                }
            }
    
        }
    
        protected override async void OnActivityResult(int requestCode, Result resultVal, Intent data)
        {
            if (requestCode == VOICE)
            {
                 clock = 0;// na 7 secs start opname opnieuw
    
                if (resultVal == Result.Ok)
                {
                    var matches = data.GetStringArrayListExtra(RecognizerIntent.ExtraResults);
                    var result = Question;
                    RunOnUiThread(() => result.Text = matches[0].ToString().ToLower());// belangrijk tolower()!
    
                    RunOnUiThread(() => Ask.PerformClick()); // runonui belangrijk in deze timegevoelige handelingen
    
                }
                else
                {
    
                    RunOnUiThread(() => Answer.Text = "Spreek wanneer er 'spreek' tevoorschijn komt op het scherm. Wacht na het spreken eventjes.");
                    RunOnUiThread(() => spreken.Eir_Stem("Spreek wanneer er 'spreek' tevoorschijn komt op het scherm. Wacht na het spreken eventjes."));
                    RunOnUiThread(() => Ask.PerformClick());
    
                }
            }
            base.OnActivityResult(requestCode, resultVal, data);
        }
    

    _

    Note: I've delete some sensitive code. You can use this example, but copy-paste won't suffice.

    If you think the code is too sloppy or somewhat wrong, feel free to enlighten me :).

    Kind regards,
    Jonathan N.

Sign In or Register to comment.