当前位置：网站首页>Wpewebkit debugging MSE playback

Wpewebkit debugging MSE playback

2022-06-25 22:44:00 【Hui's technical notes】

download mse-demo

git clone https://github.com/bitmovin/mse-demo

cog start-up

cog function , Configure environment variables for compilation each time

export PATH=$PATH:/home/hui/disk4t/codes/wpe/source/inst/bin

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/hui/disk4t/codes/wpe/source/inst/lib:/home/hui/disk4t/codes/wpe/source/inst/lib/x86_64-linux-gnu

export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/home/hui/disk4t/codes/wpe/source/inst/lib/pkgconfig:/home/hui/disk4t/codes/wpe/source/inst/lib/x86_64-linux-gnu/pkgconfig

cog --platform=x11 file:///home/hui/webkit-debug/mse-demo/index.html

stay Tools/TestWebKitAPI/Tests/WebKit/file-with-mse.html In file , It's also mse Test code , You can refer to and modify the above html, Add one more button To facilitate attach To WPEWebprocess Process debugging .

diff --git a/index.html b/index.html
index 58aa0e5..b22f048 100644
--- a/index.html
+++ b/index.html
@@ -6,12 +6,12 @@
 </head>
 <body>
   <h1>MSE Demo</h1>
-  <div>
-    <video controls width="80%"></video>
-  </div>
-
+   <div>
+   <video id="player" controls width="80%" ></video>
+   </div>
+  <button οnclick="playVideo()">play video</button>
   <script type="text/javascript">
-    (function() {
+    function playVideo() {
       var baseUrl = 'https://bitdash-a.akamaihd.net/content/MI201109210084_1/video/720_2400000/dash/';
       var initUrl = baseUrl + 'init.mp4';
       var templateUrl = baseUrl + 'segment_$Number$.m4s';
@@ -68,7 +68,7 @@
 
         xhr.send();
       }
-    })();
+    };
   </script>
 </body>
-</html>
+</html>

  <script>
    var v = document.getElementById("player");
    v.onclick = function() {
     
        if (v.playing) {
     
          v.play();
        } else {
     
          v.pause();
        }
    };
  </script>

By increasing the js Code , Direct use of video Labeled play Button code doesn't work , We need to study .

obtain journal Output

journalctl -ef | grep -E "WPEWebProcess|WPENetworkProcess"

security origin

function cog when , Find out cog The page can't be loaded after it gets up ,chrome Play normal ,log There is ：

was rejected by SecurityOrigin

Be being SecurityOrigin This place is directly return 了 , In order to test , Modify this place directly , return true, Just compile and run .

    RefPtr<Frame> frame = document().frame();
    if (!frame || !document().securityOrigin().canDisplay(url)) {
    
        if (actionIfInvalid == Complain) {
    
            FrameLoader::reportLocalLoadFailed(frame.get(), url.stringCenterEllipsizedToLength());
            ERROR_LOG(LOGIDENTIFIER, url , " was rejected by SecurityOrigin");
        }
-       return false;
+       return true;
    }

After that , adopt qtcreator Breakpoint debugging is OK , You can easily see mediasource and sourcebuffer Call stack .

mediasource The creation of

MediaSource Is in js Can only be created through code calls in , Not all html5 video All are MSE, So at the beginning of contact , use https://www.w3.org/2010/05/video/mediaevents.html Test page , There is no such process , I doubt whether the code idea is wrong .

1  WebCore::MediaSource::MediaSource
2  WebCore::MediaSource::create
3  WebCore::JSDOMConstructor<WebCore::JSMediaSource>::construct

addSourceBuffer

Again ,sourcebuffer Is also created through js Code calls go to the bottom layer C++ Code ,addSourceBuffer

1  WebCore::SourceBufferPrivateGStreamer::SourceBufferPrivateGStreamer
2  WebCore::SourceBufferPrivateGStreamer::create
3  WebCore::MediaSourcePrivateGStreamer::addSourceBuffer
4  WebCore::MediaSource::createSourceBufferPrivate
5  WebCore::MediaSource::addSourceBuffer
6  WebCore::jsMediaSourcePrototypeFunction_addSourceBufferBody
8  WebCore::jsMediaSourcePrototypeFunction_addSourceBuffer

Set by breakpoint , You can quickly see AppendPipeline The creation of the ：

1   WebCore::HTMLMediaElement::prepareToPlay  
2   WebCore::HTMLMediaElement::updatePlayState
3   WebCore::HTMLMediaElement::setReadyState    
4   WebCore::HTMLMediaElement::mediaPlayerReadyStateChanged
5   WebCore::MediaPlayer::readyStateChanged
6   WebCore::MediaPlayerPrivateGStreamerMSE::setReadyState
7   WebCore::MediaSourcePrivateGStreamer::setReadyState
8   WebCore::SourceBufferPrivateGStreamer::setReadyState
9   WebCore::SourceBuffer::sourceBufferPrivateDidReceiveInitializationSegment
10  WebCore::SourceBufferPrivate::didReceiveInitializationSegment
11  WebCore::SourceBufferPrivateGStreamer::didReceiveInitializationSegment
12  WebCore::AppendPipeline::didReceiveInitializationSegment
14  WTF::Detail::CallableWrapper<WebCore::AppendPipeline::AppendPipeline

AppendPipeline In the constructor ,no-more-pads Where the signal is connected , Directly through lambda The expression calls didReceiveInitializationSegment：

Insert picture description here

html MSE Full code

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>MSE Demo</title>
</head>
<body>
  <h1>MSE Demo</h1>
   <div>
      <video id="player" controls width="80%" ></video>
   </div>
  <button onclick="playVideo()" >play video</button>
  
  <script type="text/javascript"> function playVideo() {
       var baseUrl = 'https://bitdash-a.akamaihd.net/content/MI201109210084_1/video/720_2400000/dash/'; var initUrl = baseUrl + 'init.mp4'; var templateUrl = baseUrl + 'segment_$Number$.m4s'; var sourceBuffer; var index = 0; var numberOfChunks = 52; var video = document.querySelector('video'); if (!window.MediaSource) {
       console.error('No Media Source API available'); return; } var ms = new MediaSource(); video.src = window.URL.createObjectURL(ms); ms.addEventListener('sourceopen', onMediaSourceOpen); function onMediaSourceOpen() {
       sourceBuffer = ms.addSourceBuffer('video/mp4; codecs="avc1.4d401f"'); sourceBuffer.addEventListener('updateend', nextSegment); GET(initUrl, appendToBuffer); video.play(); } function nextSegment() {
       var url = templateUrl.replace('$Number$', index); GET(url, appendToBuffer); index++; if (index > numberOfChunks) {
       sourceBuffer.removeEventListener('updateend', nextSegment); } } function appendToBuffer(videoChunk) {
       if (videoChunk) {
       sourceBuffer.appendBuffer(new Uint8Array(videoChunk)); } } function GET(url, callback) {
       var xhr = new XMLHttpRequest(); xhr.open('GET', url); xhr.responseType = 'arraybuffer'; xhr.onload = function(e) {
       if (xhr.status != 200) {
       console.warn('Unexpected status code ' + xhr.status + ' for ' + url); return false; } callback(xhr.response); }; xhr.send(); } }; </script>
</body>
</html>