当前位置：网站首页>Webrtc audio and video capture and playback examples and mediastream media stream analysis

Webrtc audio and video capture and playback examples and mediastream media stream analysis

2022-07-02 22:54:00 【Wahaha】

WebRTC Audio and video acquisition and playback examples and MediaStream Media stream parsing

Catalog

  1. Sample code —— Turn on the camera and microphone at the same time , And display the screen and play the captured sound on the page
  2. API analysis
    1. mediaDevices
    2. MediaStream Media streaming

1. Sample code —— Turn on the camera and microphone at the same time , And display the screen and play the captured sound on the page

  1. Code 
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>WebRTC Demo</title>
</head>
<body>
<video id="local-video" autoplay playsinline></video>
<button id="showVideo"> Open audio and video </button>
</body>

<script>
    const constraints = {
    
        audio: true,
        video: {
    width: 640, height: 480}
    }

    //  The camera was turned on successfully 
    function handleSuccess(mediaStream) {
    
        const video = document.querySelector("#local-video");
        video.srcObject = mediaStream;
    }

    //  exception handling 
    function handleError(error) {
    
        console.error("getUserMedia error: " + error)
    }

    function onOpenAV(e) {
    
        navigator.mediaDevices.getUserMedia(constraints).then(handleSuccess).catch(handleError);
    }

    document.querySelector("#showVideo").addEventListener("click", onOpenAV)

</script>
</html>
  1. effect
    Insert picture description here

2. API analysis

1. mediaDevices

  1. mediaDevices yes Navigator Read-only property , Return to one MediaDevices object , This object provides connection access to media input devices such as cameras and microphones , It also includes screen sharing .
  2. grammar ：
var mediaDevices = navigator.mediaDevices;
  1. MediaDevices Is a singleton object , Members of this object can be used directly , For example, by calling navigator.mediaDevices.getUserMedia().

2. MediaStream Media streaming

  1. navigator.mediaDevices.getUserMedia() The user will be prompted to give permission to use media input , Media input produces a MediaStream（ Media streaming ）, Media stream is the carrier of information , Represents the content stream of a media device .
  2. Media streams can be collected 、 Transmission and playback , Usually, a media stream contains multiple media tracks , Such as audio track 、 Video track .
  3. Media streaming use MediaStream Interface to manage , Generally, there are several ways to obtain media streams .
    a. Get streaming objects from cameras or microphones .
    b. Get the stream object from the screen share .
    c. from canvas（HTMLCanvasElement） Get the stream object from the content .
    d. From media elements （HTMLMediaElement） Get stream object .
  4. The media streams obtained by the above methods can be obtained through WebRTC transmitted , And share among multiple peers .
  5. Return to one Promise object , After success resolve Call back one MediaStream object .
  6. If the user refuses permission , Or the required media source is not available ,promise Meeting reject Call back one PermissionDeniedError perhaps NotFoundError .
  7. Common use ：
navigator.mediaDevices.getUserMedia(constraints)
.then(function(stream) {
    
  /*  Use this  stream stream */
})
.catch(function(err) {
    
  /*  Handle  error */
});
  1. MediaStream The definition of 
interface MediaStream : EventTarget {
    
  constructor();
  constructor(MediaStream stream);
  constructor(sequence<MediaStreamTrack> tracks);
  readonly attribute DOMString id;
  sequence<MediaStreamTrack> getAudioTracks();
  sequence<MediaStreamTrack> getVideoTracks();
  sequence<MediaStreamTrack> getTracks();
  MediaStreamTrack? getTrackById(DOMString trackId);
  void addTrack(MediaStreamTrack track);
  void removeTrack(MediaStreamTrack track);
  MediaStream clone();
  readonly attribute boolean active;
  attribute EventHandler onaddtrack;
  attribute EventHandler onremovetrack;
};

1. MediaStream attribute

1. active read-only
  1. return MediaStream The state of , The type is Boolean ,true Indicates active ,false Indicates inactive .
2. id read-only
  1. return MediaStream Of UUID, Type is string , The length is 36 Characters .

2. MediaStream Method

1. addTrack() Method ： Add new media tracks to the media stream .
stream.addTrack(track);
 Parameters ：Track, Media track , The type is MediaStreamTrack.
 Return value ： nothing .
2. clone() Method ： Returns a copy of the current media stream , Copies have different and unique identifiers .
const newstream = stream.clone();
// sameId by false
const sameId = newstream.id === stream.id? true : false
 Parameters ： nothing .
 Return value ： A new media streaming object .
3. getAudioTracks() Method ： The returned media type is audio An array of media track objects , The array member type is MediaStreamTrack.
  1. Be careful , The order of the array is uncertain , Each call may be different .
const mediaStreamTracks = mediaStream.getAudioTracks()
 Parameters ： nothing .
 Return value ：mediaStreamTracks, Media track object array , If the current media stream has no audio track , The returned array is empty .
  1. Example ： Use getUserMedia() Method to obtain the media stream containing video and audio tracks , If the call succeeds , Then attach the media stream to <video> Elements , Then set the timer ,5s After the call getAudioTracks() Method to get all audio tracks , Finally, stop playing the first audio track .
navigator.mediaDevices.getUserMedia({
    audio: true, video: true})
  .then(mediaStream => {
    
  document.querySelector('video').srcObject = mediaStream;
  // 5s after , Stop playing the first audio track 
  setTimeout(() => {
    
    const tracks = mediaStream.getAudioTracks()
    tracks[0].stop()
  }, 5000)
})
4. getVideoTracks() Method ： return kind The property value is video An array of media track objects , The media track object type is MediaStream Track.
  1. Be careful , The order of objects in the array is uncertain , Each call may be different .
const mediaStreamTracks = mediaStream.getVideoTracks()
 Parameters ： nothing .
 Return value ：mediaStreamTracks Is an array of media track objects . If the current media stream has no video track , The returned array is empty .
  1. Example ：getUserMedia() Method to get the video stream , If the call succeeds , Then attach the media stream to <video> Elements , Then get the first video track and take pictures from the video track .
navigator.mediaDevices.getUserMedia({
    video: true})
  .then(mediaStream => {
    
  document.querySelector('video').srcObject = mediaStream;
  const track = mediaStream.getVideoTracks()[0];
  //  Capture pictures 
  const imageCapture = new ImageCapture(track);
  return imageCapture;
})
5. getTrackById() Method ： Returns the specified ID Track object of .
  1. If no parameters are provided , Or no match ID value , Then return to null; If there are multiple identical ID The orbit of , This method returns the first track that matches .
const track = MediaStream.getTrackById(id);
 Parameters ：id, Type is string .
 Return value ： If you enter parameters id And MediaStreamTrack.id matching , Then return the corresponding MediaStream-Track object , Otherwise return to null.
  1. Example ： Get specified ID And apply constraints , Adjust the volume to 0.5.
stream.getTrackById("primary-audio-track").applyConstraints({
     volume: 0.5 });
6. getTracks() Method ： Returns an array of all media track objects , Including all video and audio tracks .
  1. The order of objects in the array is uncertain , Each call may be different .
const mediaStreamTracks = mediaStream.getTracks()
 Parameters ： nothing .
 Return value ： Media track object array .
  1. Use getUserMedia() Method to get the stream containing the video track , If the call succeeds , Then attach the stream to <video> Elements , Then set the timer ,5s Get all media tracks after , And stop playing the first media track （ Video track ）.
navigator.mediaDevices.getUserMedia({
    audio: false, video: true})
.then(mediaStream => {
    
  document.querySelector('video').srcObject = mediaStream;
  // 5s after , Stop playing the first media track 
  setTimeout(() => {
    
    const tracks = mediaStream.getTracks()
    tracks[0].stop()
  }, 5000)
})

3. MediaStream event

1. addtrack event ： When there is a new media track （MediaStreamTrack） This event is triggered when joining , Corresponding event handle onaddtrack
  1. Be careful , Only if , Will trigger the event , Active call MediaStream.addTrack() Method does not trigger .
    1. RTCPeerConnection Renegotiate .
    2. HTMLMediaElement.captureStream() Return to the new media track .
  2. Example ： When a new media track is added to the media stream , Show the category and label of the new media track .
// event The type is MediaStreamTrackEvent
// event.track The type is MediaStreamTrack
stream.onaddtrack = (event) => {
    
  let trackList = document.getElementById("tracks");
  let label = document.createElement("li");

  label.innerHTML = event.track.kind + ": " + event.track.label;
  trackList.appendChild(label);
};
  1. Besides , You can also use addEventListener() Method monitor events addtrack.
2. removetrack event ： This event is triggered when a media track is removed , Corresponding event handle onremovetrack
  1. Be careful , This event will only be triggered if , Active call MediaStream.removeTrack() Method does not trigger .
    1. RTCPeerConnection Renegotiate .
    2. HTMLMediaElement.captureStream() Return to the new media track .
  2. Example ： When deleting a media track from a media stream , Record the media track information .
// event The type is MediaStreamTrackEvent
// event.track The type is MediaStreamTrack
stream.onremovetrack = (event) => {
    
  let trackList = document.getElementById("tracks");
  let label = document.createElement("li");

  label.innerHTML = "Removed: " + event.track.kind + ": " + event.track.label;
  trackList.appendChild(label);
};
  1. Besides , You can also use addEventListener() Method monitor events removetrack.

4. Parameters constraints

  1. constraints As a MediaStreamConstraints object , Specifies the requested media type and the corresponding parameters .

  2. constraints The parameter is a parameter that contains video and audio Two members of MediaStreamConstraints object , Used to describe the media type of the request .

    1. At least one type or both must be specified .
    2. If the browser cannot find the specified media type or meet the corresponding parameter requirements , So back Promise The object will be in rejected［ Failure ］ state ,NotFoundError As rejected［ Failure ］ Parameters of the callback .

  3. The following requests both audio and video without any parameters ：

{
     audio: true, video: true }

  1. If... Is set for a media type true , This type of orbit is needed in the resulting stream . If one of them cannot be obtained for some reason ,getUserMedia() There will be a mistake .

  2. When for privacy reasons , When the user's camera and microphone information cannot be accessed , Applications can use additional constraints Parameters request the camera and microphone capabilities it needs or wants .

  3. The following is a demonstration of what the application wants to use 1280x720 Camera resolution ：

{
    
  audio: true,
  video: {
     width: 1280, height: 720 }
}
  1. The browser will try to satisfy this request parameter , However, if the parameter requirements in the request cannot be met accurately or the user chooses to override the parameters in the request , It is possible to return to other resolutions .
  2. When a specific size is mandatory , Keywords can be used min、max perhaps exact（ Namely min == max）.
  3. The following parameters indicate that the minimum required is 1280x720 The resolution of the .
{
    
  audio: true,
  video: {
    
    width: {
     min: 1280 },
    height: {
     min: 720 }
  }
}
  1. If the camera does not support the requested or higher resolution , Back to Promise Will be rejected state ,NotFoundError As rejected Parameters of the callback , And users will not be prompted to ask for authorization .
  2. The reason for the different performance is , Compared with simple request values and ideal Key words , keyword min, max, and exact Compulsion with internal relevance , such as ：
{
    
  audio: true,
  video: {
    
    width: {
     min: 1024, ideal: 1280, max: 1920 },
    height: {
     min: 776, ideal: 720, max: 1080 }
  }
}
  1. When the request contains a ideal（ Application ideal ） When the value of , This value has a higher weight , It means that the browser will first try to find the setting or camera closest to the specified ideal value （ If the device has more than one camera ）.
  2. The simple request value can also be understood as the ideal value of the application , Therefore, our first request to specify the resolution can also be written as follows ：
{
    
  audio: true,
  video: {
    
    width: {
     ideal: 1280 },
    height: {
     ideal: 720 }
  }
}
  1. Not all constraints It's all numbers . for example , On the mobile device , The following example shows that the front camera is preferred （ If any ）：
{
     audio: true, video: {
     facingMode: "user" } }
  1. Force the use of rear Cameras , Please use ：
{
     audio: true, video: {
     facingMode: {
     exact: "environment" } } }
  1. In some cases , such as WebRTC When using limited bandwidth transmission on , Low frame rate may be more appropriate .
{
     video: {
     frameRate: {
     ideal: 10, max: 15 } } };

5. Return value 

var promise = navigator.mediaDevices.getUserMedia(constraints);
  1. Return to one Promise, This Promise Successful callback function with a MediaStream Object as its parameter .

6. outliers

  1. Returns a status of failure Promise, This Promise The callback function after failure takes a DOMException Object as its parameter . Possible exceptions are ：
  2. AbortError［ Abort error ］
    1. Although both the user and the operating system grant access to the device hardware , And there is no possibility of throwing NotReadableError Abnormal hardware problems , But there are still some problems that make the device unusable .
  3. NotAllowedError［ Reject error ］
    1. The user rejected the access request of the current browser instance ; Or the user denied access to the current session ; Or the user has rejected all media access requests globally .
    2. Older versions of the specification used SecurityError, But in the new version SecurityError Given new meaning .
  4. NotFoundError［ No errors found ］
    1. Cannot find media type that meets the requested parameters .
  5. NotReadableError［ Unable to read error ］
    1. Although the user has been authorized to use the corresponding device , Some hardware on the operating system 、 An error at the browser or web level makes the device inaccessible .
  6. OverconstrainedError［ Unable to meet requirements error ］
    1. The specified requirements cannot be met by the device , This exception is of type OverconstrainedError The object of , To have a constraint attribute , This property contains the currently unsatisfiable constraint object , And have a message attribute , Contains a reading friendly string to illustrate the situation .
  7. NotFoundError［ No errors found ］
    1. Cannot find media type that meets the requested parameters .
  8. NotReadableError［ Unable to read error ］
    1. Although the user has been authorized to use the corresponding device , Some hardware on the operating system 、 An error at the browser or web level makes the device inaccessible .
  9. OverconstrainedError［ Unable to meet requirements error ］
    1. The specified requirements cannot be met by the device , This exception is of type OverconstrainedError The object of , To have a constraint attribute , This property contains the currently unsatisfiable constraint object , And have a message attribute , Contains a reading friendly string to illustrate the situation .
  10. SecurityError［ Security error ］
    1. stay getUserMedia() The called Document above , Using device media is forbidden . Whether this mechanism is turned on or off depends on the preferences of individual users .
  11. TypeError［ Type error ］
    1. constraints Object not set ［ empty ］, Or both are set to false.
原网站

版权声明
本文为[Wahaha]所创，转载请带上原文链接，感谢
https://yzsam.com/2022/183/202207022031173155.html

边栏推荐

猜你喜欢

随机推荐