Wkwebview audio and video media playback processing

1. Yes WKWebViewConfiguration Set it up .

Media files can be played automatically 、 Use embedded HTML5 Play and other functions Use this Test website

//  Initialize the configuration object 
WKWebViewConfiguration *configuration = [[WKWebViewConfiguration alloc] init];
//  The default is NO, This value determines the use of inline HTML5 Play video with local full screen control 
configuration.allowsInlineMediaPlayback = YES;
//  Auto play ,  The user is not required to take any gesture to start playing 
// WKAudiovisualMediaTypeNone  The playback of audio and video does not need to be triggered by user gestures ,  Auto play 
configuration.mediaTypesRequiringUserActionForPlayback = WKAudiovisualMediaTypeNone;
configuration.allowsAirPlayForMediaPlayback = YES;
configuration.allowsPictureInPictureMediaPlayback = YES;
    
self.webView = [[WKWebView alloc] initWithFrame:self.view.bounds configuration:configuration];
self.webView.navigationDelegate = self;
NSURL *url =[NSURL URLWithString:@" Test website "];
[self.webView loadRequest:[NSURLRequest requestWithURL:url]];
[self.view addSubview:self.webView];

because H5 Of video Not set autoplay、playsinline attribute . We need to inject ourselves , In order to achieve the effect .

NSString *jSString = @"document.getElementsByTagName('video')[0].setAttribute('playsinline','');";
NSString *jSString2 = @"document.getElementsByTagName('video')[0].autoplay=true;";
// Used for JavaScript Inject 
WKUserScript *wkUScript = [[WKUserScript alloc] initWithSource:jSString injectionTime:WKUserScriptInjectionTimeAtDocumentEnd forMainFrameOnly:YES];
WKUserScript *wkUScript2 = [[WKUserScript alloc] initWithSource:jSString2 injectionTime:WKUserScriptInjectionTimeAtDocumentEnd forMainFrameOnly:YES];
[configuration.userContentController addUserScript:wkUScript];
[configuration.userContentController addUserScript:wkUScript2];

2. Listen for the callback of the player in the web page

There are two ways .

2.1 utilize HTML5 Audio/Video event

HTML5 Audio/Video Event codes can be defined by H5 Colleague complete , Also can be App End injection . The injection code is as follows ：

NSString *jSString3 = @"document.getElementsByTagName('video')[0].addEventListener('canplay', function(e) {window.webkit.messageHandlers.readytoplay.postMessage(\"canplay\");})";
NSString *jSString4 = @"document.getElementsByTagName('video')[0].addEventListener('pause', function(e) {window.webkit.messageHandlers.pause.postMessage(\"pause\");})";
NSString *jSString5 = @"document.getElementsByTagName('video')[0].addEventListener('play', function(e) {window.webkit.messageHandlers.play.postMessage(\"play\");})";
NSString *jSString6 = @"document.getElementsByTagName('video')[0].addEventListener('ended', function(e) {window.webkit.messageHandlers.ended.postMessage(\"ended\");})";
WKUserScript *wkUScript3 = [[WKUserScript alloc] initWithSource:jSString3 injectionTime:WKUserScriptInjectionTimeAtDocumentEnd forMainFrameOnly:YES];
[configuration.userContentController addUserScript:wkUScript3];
WKUserScript *wkUScript4 = [[WKUserScript alloc] initWithSource:jSString4 injectionTime:WKUserScriptInjectionTimeAtDocumentEnd forMainFrameOnly:YES];
[configuration.userContentController addUserScript:wkUScript4];
WKUserScript *wkUScript5 = [[WKUserScript alloc] initWithSource:jSString5 injectionTime:WKUserScriptInjectionTimeAtDocumentEnd forMainFrameOnly:YES];
[configuration.userContentController addUserScript:wkUScript5];
WKUserScript *wkUScript6 = [[WKUserScript alloc] initWithSource:jSString6 injectionTime:WKUserScriptInjectionTimeAtDocumentEnd forMainFrameOnly:YES];
[configuration.userContentController addUserScript:wkUScript6];

App End receiving js The code for is as follows ： To be observed WKScriptMessageHandler agreement

@interface ViewController () <WKNavigationDelegate,WKScriptMessageHandler>
@end

Again WKWebViewConfiguration add protocols

// Add a protocol 
[configuration.userContentController addScriptMessageHandler:self name:@"readytoplay"];
[configuration.userContentController addScriptMessageHandler:self name:@"play"];
[configuration.userContentController addScriptMessageHandler:self name:@"pause"];
[configuration.userContentController addScriptMessageHandler:self name:@"ended"];

Use the following methods to get player events

#pragma mark - WKScriptMessageHandler

//! WKWebView received ScriptMessage Call back this method 
- (void)userContentController:(WKUserContentController *)userContentController didReceiveScriptMessage:(WKScriptMessage *)message {
    if ([message.name caseInsensitiveCompare:@"readytoplay"] == NSOrderedSame) {
        NSLog(@"video is readytoplay");
    }
    if ([message.name caseInsensitiveCompare:@"play"] == NSOrderedSame) {
        NSLog(@"video is play");
    }
    if ([message.name caseInsensitiveCompare:@"pause"] == NSOrderedSame) {
        NSLog(@"video is pause");
    }
    if ([message.name caseInsensitiveCompare:@"ended"] == NSOrderedSame) {
        NSLog(@"video is ended");
    }
}

Reference material ： HTML Audio / Video reference manual video Complete usage of attributes and events iOS And JS Interactive WKWebView-WKScriptMessageHandler agreement

2.2 The other is App It can be realized by itself , Use AVAudioSession monitor ：

Use AVAudioSession monitor , Must be used AVAudioSessionCategoryOptionMixWithOthers. This will cause switching between other audio and video App Will not interrupt the player . For example, Netease cloud music 、bilibili. Phone calls interrupt the player .

NSError *sessionError = nil;
[[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryPlayback
                                 withOptions:AVAudioSessionCategoryOptionMixWithOthers
                                       error:&sessionError];
[[AVAudioSession sharedInstance] setActive:YES error:nil];
[[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(audioSessionSilenceSecondaryAudioHint:)
                                             name:AVAudioSessionSilenceSecondaryAudioHintNotification
                                           object:[AVAudioSession sharedInstance]];

- (void)audioSessionSilenceSecondaryAudioHint:(NSNotification *)notification
{
    NSDictionary *userInfo = notification.userInfo;
    NSLog(@"audioSessionSilenceSecondaryAudioHint  %@",userInfo);
}

Start playing output ：

2021-04-01 15:22:31.302248+0800 webViewPlayMedia[18078:2811391] audioSessionSilenceSecondaryAudioHint  {
    AVAudioSessionSilenceSecondaryAudioHintTypeKey = 1;

End playback output ：

2021-04-01 15:22:31.382646+0800 webViewPlayMedia[18078:2811391] audioSessionSilenceSecondaryAudioHint  {
    AVAudioSessionSilenceSecondaryAudioHintTypeKey = 0;

3. Get video playback address , Play with a custom player

- (void)webView:(WKWebView *)webView didFinishNavigation:(null_unspecified WKNavigation *)navigation {
    NSLog(@"WKPhoneWebView didFinishNavigation");
    
    NSString *JsStr = @"(document.getElementsByTagName(\"video\")[0]).src";
    [self.webView evaluateJavaScript:JsStr completionHandler:^(id _Nullable response, NSError * _Nullable error) {
        if(![response isEqual:[NSNull null]] && response != nil){
            // The video address has been intercepted 
            NSLog(@"response == %@",response);
        }else{
            // No video link 
        }
    }];
}

4. pit

4.1 Play the video , There will be ERROR Tips ：

2021-04-01 09:34:57.361477+0800 webViewPlayMedia[17109:2655981] [assertion] Error acquiring assertion: <Error Domain=RBSAssertionErrorDomain Code=3 "Required client entitlement is missing" UserInfo={RBSAssertionAttribute=<RBSDomainAttribute| domain:"com.apple.webkit" name:"MediaPlayback" sourceEnvironment:"(null)">, NSLocalizedFailureReason=Required client entitlement is missing}>
2021-04-01 09:34:57.361610+0800 webViewPlayMedia[17109:2655981] [ProcessSuspension] 0x1043dc990 - ProcessAssertion: Failed to acquire RBS MediaPlayback assertion 'WebKit Media Playback' for process with PID 17110, error: Error Domain=RBSAssertionErrorDomain Code=3 "Required client entitlement is missing" UserInfo={RBSAssertionAttribute=<RBSDomainAttribute| domain:"com.apple.webkit" name:"MediaPlayback" sourceEnvironment:"(null)">, NSLocalizedFailureReason=Required client entitlement is missing}

But it's set background The attribute is , Still unable to remove , But it does not affect the playback . The question is https://stackoverflow.com/questions/66493177/required-client-entitlement-is-missing-in-wkwebview It was also suggested that , But there is no solution .

4.2 iOS13.2 13.3 The system phone will load WKWebView It will continuously report errors ：

2021-04-01 15:55:11.083253+0800 webViewPlayMedia[342:59346] [Process] kill() returned unexpected error 1

Under this system version ,WKWebView Use configuration WKWebViewConfiguration, Will not play .

Information ： Console warning received ： When I was there iOS13.2 Load in WKWebView when ,[Process] kill() returned unexpected error 1