[freeswitch development practice] media bug obtains call voice flow

Blog home page ： The home page of the pony driver
My column ：FreeSwitch Development practices
Column introduction ： It mainly introduces the use of bloggers in actual projects FreeSwitch Some experience of developing outbound call projects , Mainly involves FreeSwitch Basic installation and compilation of 、 Basic configuration 、ESL、WSS、 sound recording 、 Custom module 、media bug、 Voice playback 、MRCP And docking AI Robots and so on . The content is constantly updated , If you are interested, you can subscribe to the column ~

What you want to say to yourself in the future

Intermittent efforts and muddle through , It's all about clearing the previous efforts

---

Preface

Last introduction Create and use custom modules , Create your own app Embedded in FreeSwitch; This article introduces the use of “media bug”, Get the call voice stream . First , See here , There should be two problems ：

• What is? media bug
• media bug What's the usage? , Can do
  The first question will be introduced in detail later , The second question is media bug Can do ,media bug It is mainly used to monitor voice data , And you can do a lot of things when you get the call voice stream , You can add a lot of business ：
• Call monitor
• Analyze voice , That is to say ASR（ speech recognition ）

---

One 、 What is? media bug

media bug Medium bug It doesn't mean software defects , Its function is to FreeSwitch Media call data flow ( Voice or video ) Intercept this part of the data stream , Achieve the function of monitoring , Similar to inserting a pipe from a pipe , This pipe is “media bug”, It's equivalent to hitting a “ hole ”, therefore media bug Of bug It should be called “ Stealing ”、“ Intercept ” It means .

Two 、 How to use media bug Get the call voice stream

First , Look at a paragraph media bug An example of creating and obtaining a call voice stream ：

2.1 media bug establish

switch_status_t status;
switch_media_bug_t *bug;
// add to media bug
status = switch_core_media_bug_add(session, "mymedia", 0, mymediabug_callback, (void*)robot_id, 0, SMBF_READ_STREAM | SMBF_NO_PAUSE, &bug);

Let's see switch_core_media_bug_add Statement of ：

SWITCH_DECLARE(switch_status_t) switch_core_media_bug_add(_In_ switch_core_session_t *session,
														  _In_ const char *function,
														  _In_ const char *target,
														  _In_ switch_media_bug_callback_t callback,
														  _In_opt_ void *user_data,
														  _In_ time_t stop_time, 
														  _In_ switch_media_bug_flag_t flags, _Out_ switch_media_bug_t **new_bug);

Function description ：

session	conversation session
function	media bug The name of the function , Can be the only key, One key Corresponding to one media bug, Corresponding Media-Bug-Function
target	The goal is , Corresponding Media-Bug-Taget
callback	media bug Callback function
user_data	Incoming data of callback function
stop_time	Stop time
flags	sign , No flag , Functions that fail to listen in callback functions
new_bug	Out parameter , New creation media bug

2.2 media bug Callback function

static switch_bool_t mymediabug_callback(switch_media_bug_t *bug, void *user_data, switch_abc_type_t type)
{
    
	switch_core_session_t* session = switch_core_media_bug_get_session(bug);
    uint8_t data[SWITCH_RECOMMENDED_BUFFER_SIZE];
    switch_frame_t frame = {
     0 };
    frame.data = data;
    frame.buflen = sizeof(data);

    const char*robot_id = (const char *) user_data;
    if (robot_id == NULL) {
    
        switch_log_printf(SWITCH_CHANNEL_LOG, SWITCH_LOG_WARNING, "mymedia user data is null!");
        return SWITCH_FALSE;
    }
    
    switch (type) {
    
            
        case SWITCH_ABC_TYPE_INIT:
        {
    
			switch_log_printf(SWITCH_CHANNEL_LOG, SWITCH_LOG_WARNING, "mymediabug init, user_data[%s]\n", robot_id);
        }
        break;
    
        case SWITCH_ABC_TYPE_READ:
        {
    
            if (switch_core_media_bug_read(bug, &frame, SWITCH_FALSE) != SWITCH_STATUS_FALSE) {
    
				switch_log_printf(SWITCH_CHANNEL_LOG, SWITCH_LOG_WARNING, "read frame: data_len=[%d], rate[%d], channels[%d], samples[%d] \n",
					frame.datalen, frame.rate, frame.channels, frame.samples);
             }
        }
	
		break;
        case SWITCH_ABC_TYPE_CLOSE:
			switch_log_printf(SWITCH_CHANNEL_LOG, SWITCH_LOG_WARNING, "mymediabug close, user_data[%s]\n", robot_id);
            break;
        default:
            break;
    }
    
    return SWITCH_TRUE;
}

explain ：

SWITCH_ABC_TYPE_READ This is where the voice stream is obtained ,switch_core_media_bug_read That is, the interface to obtain voice data , Let's take a look at the declaration of this interface ：

SWITCH_DECLARE(switch_status_t) switch_core_media_bug_read(_In_ switch_media_bug_t *bug, _In_ switch_frame_t *frame, switch_bool_t fill);

This interface parameter is relatively simple , bug Namely media bug The pointer , frame It is a voice data structure

2.3 media bug Principle

There is no explanation here media bug The underlying implementation of , Just briefly media bug The implementation of the . If you want to know media bug From the interface switch_core_media_bug_add Starting with , Actually media bug No new threads have been started , From the source code, it's just a video video Open thread , And audio won't , How does it monitor the audio stream , The answer is switch_core_session_read_frame Interface , This interface reads the original voice stream , Use in the core , This is why voice streams do not have separate threads .

3、 ... and 、media bug Related interfaces

The last section knows media bug establish , And in media bug The method of getting voice stream in callback function , The following lists and media bug Related interfaces ：

• switch_core_media_bug_add -- establish media bug
• switch_core_media_bug_remove – Delete media bug
• switch_core_media_bug_get_session – obtain media bug The call to which it belongs session
• switch_core_media_bug_read – from media bug Read voice data in

Four 、 Complete example

Due to code comparison , For a complete example, see ：https://download.csdn.net/download/xxm524/86261083
( Code examples symbolically charge 5 Integral points , If there are no points, bloggers can get them for free )
See ：【FreeSwitch Development practices 】 Custom module creation and use
command ：

originate user/1000 123

Output ：

---

summary

This paper mainly introduces media bug Relevant knowledge , There are the following 3 spot ：

• Explains what is media bug
• How to use media bug Get the call voice stream
• media bug Relevant interface description

If you feel some help or think the article is good , please Focus on What about bloggers , Your attention is the driving force of my continuous writing . in addition , If there are any questions , Leave a comment in the comments section , Or private bloggers , Bloggers will reply as soon as they see it .