[play Tencent cloud] experience and development of game multimedia engine (II)

Through the game multimedia engine sdk Make your own real-time voice to text .

How to enable ?

Search in Tencent cloud " Game multimedia engine ", first , It will be added to the console after opening ,

We will use it directly this time sdk, No more going to see demo, Download his SDK To use .

So we just download this sdk, Import to unity In the to

Be careful , Here is sdk download , No demo 了 . After downloading, you need to unzip it , Create a new one unity engineering , Direct will sdk Put it in Asset Under the folder .

We will build our own UI Interface , Look at the picture below , If you can't build it , Suggest unity Enter the door .

then , Create a new script for writing functions , This script is used to log in the user , After pressing the button, you can upload the sound clip and display the recognized text on the interface . good , We also need to build a new one controller The object is used to mount the script : The hierarchy and format are shown in the following figure :

Open the script , The first step is to initialize SDK. The code is as follows :

int ret = ITMGContext.GetInstance().Init(sdkAppId, openID);
// Determine whether the initialization is successful by the return value 
if (ret != QAVError.OK)
    {
        Debug.Log("SDK initialization failed :"+ret);
        return;
    }
 else 
    {
    Debug.Log (" Successful initialization !");
    }
    

This is written during script initialization , That means you can write in Awake Can also be written in Start in , As long as it can be initialized . We also need to state Appid and OpenID, We declare directly that Public, And then directly assign .

public string sdkAppid;
public string openID;
public string AuthKey;

Then assign values directly in the panel :

Do it here ,SDK Initialization is complete , That means , We can develop the actual function , Connected to Tencent cloud SDK That's all I do . The cost of learning is relatively small . I suggest you take a look at this .

Then we test and run , It is found that the instantiation has been successful :

Then we have to demo One of them is called :EnginePollHelper.cs Put it into the project . Because after using this script , Many functions don't need to be written by ourselves , A lot of things have been saved .

We need to build a new one userConfig.cs Script , This script is used to record some login related information . So this userConfig No need to mount , Just a record class , So don't inherit from Monobehaviour.

class UserConfig
{
    public static string GetExperientialAppID()
    {
        return GetAppID();
    }

    public static string GetExperientialauthkey()
    {
        return GetAuthKey();
    }

    public static bool GetTestEnv() {
		return PlayerPrefs.GetInt("TestEnv", 0) != 0;
	}
	
	public static void SetTestEnv(bool test) {
		PlayerPrefs.SetInt("TestEnv", test ? 1 : 0);
	}
     
    public static string GetAppID() {
        return PlayerPrefs.GetString("AppID", " On your own console appid");
	}

    public static string GetAuthKey()
    {
        return PlayerPrefs.GetString("AuthKey", " The key on your own console ");
    }

    public static void SetAuthKey(string AuthKey)
    {
        PlayerPrefs.SetString("AuthKey", AuthKey);
    }

    public static void SetAppID(string appID) {
		PlayerPrefs.SetString("AppID", appID);
	}

	public static string GetUserID() {
		int randomUId = UnityEngine.Random.Range(12345, 22345);
		return PlayerPrefs.GetString("UserID", randomUId.ToString() );
	}

	public static void SetUserID(string userID) {
		PlayerPrefs.SetString("UserID", userID);
	}


    public static string GetRoomID() {
		return PlayerPrefs.GetString("strRoomID", "banana");
	}

	public static void SetRoomID(string roomID) {
		PlayerPrefs.SetString("strRoomID", roomID);
	}

	public static ITMGRoomType GetRoomType() {
		return (ITMGRoomType)PlayerPrefs.GetInt("RoomType", 1);
	}

	public static void SetRoomType(ITMGRoomType roomtype) {
		PlayerPrefs.SetInt("RoomType", (int)roomtype);
	}

	public static byte[] GetAuthBuffer(string sdkAppID, string userID, string roomID,string authKey)
	{
        string key = "";
        key = authKey;
        return QAVAuthBuffer.GenAuthBuffer(int.Parse(sdkAppID), roomID, userID, key);
	}
}

Put it in , We also need to introduce a namespace in these scripts :"TencentMobileGaming".

Come back to test Script , It also needs star Write in :

UserConfig.SetAppID(sdkAppid );
UserConfig.SetUserID(openID);
UserConfig.SetAuthKey(AuthKey);
 byte[] authBuffer = UserConfig.GetAuthBuffer(UserConfig.GetAppID(), UserConfig.GetUserID(), null,UserConfig.GetAuthKey());

That means , No data has been sent to Tencent cloud , Will bring this information , The server performs a verification , It can be understood in such a simple way .

Then the most important sentence of code :

ITMGContext.GetInstance ().GetPttCtrl ().ApplyPTTAuthbuffer(authBuffer);

Another sentence is to check whether the microphone on the client is enabled , It's also written in start In the method

 int retCode = (int)ITMGContext.GetInstance ().CheckMicPermission ();
        Debug.Log (string.Format ("Check permission Code is {0}",retCode));

To finish these , Let's try to run :

This means that our code is working properly , Now let's write some functions of pressing the button to start recording and stop recording .

Let's first write a declaration of the two buttons and their functions :

public Button btn_speak;
public Button btn_stop;

public void btn_SpeakClick ()
{
btn_speak.GetCompontInChildren<Text>.text= " Recording ...";
}
public void btn_StopClick ()
{
btn_stop.GetCompontInChildren<Text>.text = " Start the recording ";
}

Then assign values to the buttons in the panel , And register the method in the click event of the button

The above sentence is very basic , If you don't understand the advice unity Enter a door .

well , Next, you need to write specific methods in this button . First, start recording , Because this is recording while talking , therefore , reference demo Writing in Chinese , Our setting here should be " streaming ", The language is Putonghua in Chinese Mainland . The temporary storage address should also be set , So it needs to be written in the code :

   string speechLanguage = "cmn-Hans-CN";// This represents Mandarin  ( The Chinese mainland )

  string recordPath = Application.persistentDataPath + string.Format("/{0}.silk", sUid++);
  if (ret != 0)
		{
			OnStreamingRecComplete(-1, "","","");
		}

This is the local address that represents the temporary existence of streaming recording ,

The recording is not sent directly to Tencent cloud , The first is the existence of local , And then through memory , To the Internet .

And then write down OnStreamingRecComplete Methods :

void OnStreamingRecComplete(int code, string fileid, string filePath, string result){
		if (code == 0)
		{ 
			field.text = result;
			Debug.Log(" Recording complete ");
		}
		else if(code == -1)
		{
			Debug.Log(" Recording failed : Streaming is recording " );
		}
		else
		{ 
			if (code == 4103)
			{
				Debug.Log(" Recording time is too short ");
			}
			else if (code == 32775)
			{
				Debug.Log("  Upload and translation failed but recording succeeded "); 
				field.text = filePath;
			}
			else if (code == 32777)
			{
				Debug.Log("  Translation failed but recording and uploading succeeded ");
				field.text = fileid;

			}
			else
			{
				Debug.Log(" Recording failed :" + Convert.ToString(code));
			}
		}
	}

This means that at each stage of streaming voice recording, a section of the server's judgment will be returned , If there is no problem with streaming data, the specific recognized text will be returned .

Then complete the function of the stop recording button .

 ITMGContext.GetInstance().GetPttCtrl().StopRecording();

Such a , It seems that the function has been written , But not yet .

You also need to write 2 A delegate method .

ITMGContext.GetInstance().GetPttCtrl().OnStreamingSpeechComplete += new QAVStreamingRecognitionCallback (OnStreamingRecComplete);
        ITMGContext.GetInstance().GetPttCtrl().OnStreamingSpeechisRunning += new QAVStreamingRecognitionCallback (OnStreamingRecisRunning);

Write the delegate method completely

(int code, string fileid, string filePath, string result){
		if (code == 0)
		{
			 
			field.text = result;
			
			Debug.Log(" Recording complete ");
		}
		else if(code == -1)
		{
			
			Debug.Log(" Recording failed : Streaming is recording " );
		}
		else
		{
			Debug.Log(mStreamBtn, " streaming ");

			if (code == 4103)
			{
				Debug.Log(" Recording time is too short ");
			}
			 
			 
			else
			{
				
				Debug(" Recording failed :" + Convert.ToString(code));
			}
		}
	}

void OnStreamingRecisRunning(int code, string fileid, string filePath, string result){
        if (code == 0)
        { 
            field.text = result; 
        }
        
    }

Okay , Come here , The function has been written , We can now try to run , Press start recording to see the real-time text on the interface :