AI target segmentation capability for fast video cutout without green screen

Green screen cutout is a common technical means in the process of film and television production,Often used in the video to dig and replace the background,More possibilities for video editing with post-processing.然而,The production cost of green screen matting technology is time-consuming and labor-intensive,cannot be used in daily life.

华为视频编辑服务Recently launched target segmentation capability,可通过AIIntelligent matting refines segmentation of target objects in video,and is not limited to a specific object class,clear in the subject、Segmentation of subject and background in videos with relatively simple backgrounds,Can get good processing effect.

应用场景

The target segmentation capability is mostly used in live video、在线教育、Scenarios such as forums and meetings.比如,In the scene of live sales,Background will broadcast can replace them by picture of the product details page,It is convenient for users to know the products in time;In online forums or video conferences,replace the background withPPTContent or office scene,Create a serious atmosphere for study and office.同样,The target segmentation ability can also meet the user's daily interesting editing,Users can edit portrait videos with target segmentation capabilities,Then switch the background to famous scenery of various countries,It can be achieved without leaving home“周游世界”.

In addition to replace the video background,Target segmentation capability can also be applied to video playback,Cutout by target,Avoid the bullet screen blocking the main body of the plot,Optimize user viewing experience.Combine other technical capabilities,The segmented target object can also be copied and deleted,Even adjust the action time of the target object,Create more creative video productions.

实现原理

那么,How is the target segmentation capability achieved?？

首先,The user needs to select the target object,然后再进行AI目标分割.The corresponding mask of the target object needs to be segmented by the first frame image of the given video,AIModel automatically in the subsequent each frame of video frames and try to match the object in the partition.并且,The model will save the result mask with good segmentation effect of the intermediate frame combined with the mask information of the first frame,Match in subsequent video frames,Can accurately outline the edge details of the target object,Further improve the accuracy of target segmentation.

DEMO演示

集成方式

1. 开发准备

详细准备步骤可参考华为开发者联盟官网

2. 编辑工程集成

2.1 设置应用的鉴权信息

可以通过api_key或者Access Token来设置应用鉴权信息.

通过setAccessToken方法设置Access Token,在应用启动时初始化设置一次即可,无需多次设置.

MediaApplication.getInstance().setAccessToken("your access token");

通过setApiKey方法设置api_key,在应用启动时初始化设置一次即可,无需多次设置.

MediaApplication.getInstance().setApiKey("your ApiKey");

2.2设置唯一标识ID,即License ID.

License ID是进行管控的有效凭证,您要保证设置License ID的唯一性.

MediaApplication.getInstance().setLicenseId("License ID");

2.2.1初始化Editor运行环境

创建编辑工程,需要首先创建Editor对象并初始化其运行环境.当离开编辑工程时,应释放Editor实例.

(1) 创建Editor对象

HuaweiVideoEditor editor = HuaweiVideoEditor.create(getApplicationContext());

(2) 指定预览窗口的布局位置

预览窗口负责视频图像画面的渲染,由视频编辑原子能力SDK内部创建SurfaceView来实现.在创建窗口之前,需要在您的App中指定预览窗口的布局位置.

<LinearLayout    
    android:id="@+id/video_content_layout"    
    android:layout_width="0dp"    
    android:layout_height="0dp"    
    android:background="@color/video_edit_main_bg_color"    
    android:gravity="center"    
    android:orientation="vertical" />
// 指定预览窗口 
LinearLayout mSdkPreviewContainer = view.findViewById(R.id.video_content_layout);

// 设置预览窗口承载的布局 
editor.setDisplay(mSdkPreviewContainer);

(3) 初始化运行环境,如果License鉴权失败,会抛出LicenseException.

当Editor对象创建之后,此时还没有占用实际的系统资源,需要手动选择其环境初始化的时机,此时视频编辑原子能力SDK内部会创建必须的线程和定时器等.

try {
        editor.initEnvironment();
   } catch (LicenseException error) { 
        SmartLog.e(TAG, "initEnvironment failed: " + error.getErrorMsg());    
        finish();
        return;
   }

3. “目标分割”能力集成

// Initialize the objectAI引擎
videoAsset.initSegmentationEngine(new HVEAIInitialCallback() {
        @Override
        public void onProgress(int progress) {
            // 初始化进度
        }

        @Override
        public void onSuccess() {
            // 初始化成功
        }

        @Override
        public void onError(int errorCode, String errorMessage) {
            // 初始化失败
    }
});

// 初始化成功后,Select the target to be segmented for segmentation,Return to choose target segmentation processing results
// bitmapA video frame picture containing the target to be segmented;timeStampFor video frame picture time stamp on the time line;pointsIt is a collection of coordinate points based on video frame pictures,左上角为原点坐标,The coordinate points should be located within the target to be segmented,And the number is recommended to be greater than or equal to2,Based on the trajectory coordinates to determine the choice
int result = videoAsset.selectSegmentationObject(bitmap, timeStamp, points);

// After the processing result of the target segmentation is successful,Add target segmentationAI特效
videoAsset.addSegmentationEffect(new HVEAIProcessCallback() {
        @Override
        public void onProgress(int progress) {
            // 目标分割AI特效处理进度
        }

        @Override
        public void onSuccess() {
            // 目标分割AI特效处理成功
        }

        @Override
        public void onError(int errorCode, String errorMessage) {
            // 目标分割AI特效处理失败
        }
});

// interrupt target segmentationAI特效处理
videoAsset.interruptSegmentation();

// remove target segmentationAI特效
videoAsset.removeSegmentationEffect();

// release target splitAI引擎
videoAsset.releaseSegmentationEngine();

了解更多详情>>

访问华为开发者联盟官网
获取开发指导文档
华为移动服务开源仓库地址：GitHub、Gitee

关注我们,第一时间了解 HMS Core 最新技术资讯~