【owt】WebrtcNode, subscribe-sdp offer 流程(1)

sdp offer 流程

1. AmqpClient - New message received

sdp offer 的消息

2023-04-26T21:54:19.790  - DEBUG: AmqpClient - RpcServer New message received {
  method: 'onTransportSignaling',
  args: [
    'b149e44bb10d4e91bd162a8c6806ae7b',
    {
      sdp: 'v=0\r\n' +
        'o=- 7177131362423164715 2 IN IP4 127.0.0.1\r\n' +
        's=-\r\n' +
        't=0 0\r\n' +
        'a=group:BUNDLE 0 1\r\n' +
        'a=msid-semantic: WMS\r\n' +
        'm=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 102 0 8 106 105 13 110 112 113 126\r\n' +
        'c=IN IP4 0.0.0.0\r\n' +
        'a=rtcp:9 IN IP4 0.0.0.0\r\n' +
        'a=ice-ufrag:h7T4\r\n' +
        'a=ice-pwd:F0L/DBkHlQzxDgFOclWm8vB7\r\n' +
        'a=ice-options:trickle\r\n' +
        'a=fingerprint:sha-256 5D:A4:FF:F1:C1:1B:51:19:CB:26:53:B6:49:2E:97:5F:F1:A4:B7:C7:41:68:AF:19:8D:FA:B1:D6:9D:02:60:4C\r\n' +
        'a=setup:actpass\r\n' +
        'a=mid:0\r\n' +
        'a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level\r\n' +
        'a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\n' +
        'a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\n' +
        'a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid\r\n' +
        'a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id\r\n' +
        'a=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id\r\n' +
        'a=recvonly\r\n' +
        'a=rtcp-mux\r\n' +
        'a=rtpmap:111 opus/48000/2\r\n' +
        'a=rtcp-fb:111 transport-cc\r\n' +
        'a=fmtp:111 minptime=10;useinbandfec=1\r\n' +
        'a=rtpmap:103 ISAC/16000\r\n' +
        'a=rtpmap:104 ISAC/32000\r\n' +
        'a=rtpmap:9 G722/8000\r\n' +
        'a=rtpmap:102 ILBC/8000\r\n' +
        'a=rtpmap:0 PCMU/8000\r\n' +
        'a=rtpmap:8 PCMA/8000\r\n' +
        'a=rtpmap:106 CN/32000\r\n' +
        'a=rtpmap:105 CN/16000\r\n' +
        'a=rtpmap:13 CN/8000\r\n' +
        'a=rtpmap:110 telephone-event/48000\r\n' +
        'a=rtpmap:112 telephone-event/32000\r\n' +
        'a=rtpmap:113 telephone-event/16000\r\n' +
        'a=rtpmap:126 telephone-event/8000\r\n' +
        'm=video 9 UDP/TLS/RTP/SAVPF 123 114 115 116 119\r\n' +
        'c=IN IP4 0.0.0.0\r\n' +
        'a=rtcp:9 IN IP4 0.0.0.0\r\n' +
        'a=ice-ufrag:h7T4\r\n' +
        'a=ice-pwd:F0L/DBkHlQzxDgFOclWm8vB7\r\n' +
        'a=ice-options:trickle\r\n' +
        'a=fingerprint:sha-256 5D:A4:FF:F1:C1:1B:51:19:CB:26:53:B6:49:2E:97:5F:F1:A4:B7:C7:41:68:AF:19:8D:FA:B1:D6:9D:02:60:4C\r\n' +
        'a=setup:actpass\r\n' +
        'a=mid:1\r\n' +
        'a=extmap:14 urn:ietf:params:rtp-hdrext:toffset\r\n' +
        'a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\n' +
        'a=extmap:13 urn:3gpp:video-orientation\r\n' +
        'a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\n' +
        'a=extmap:12 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay\r\n' +
        'a=extmap:11 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type\r\n' +
        'a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing\r\n' +
        'a=extmap:8 http://tools.ietf.org/html/draft-ietf-avtext-framemarking-07\r\n' +
        'a=extmap:9 http://www.webrtc.org/experiments/rtp-hdrext/color-space\r\n' +
        'a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid\r\n' +
        'a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id\r\n' +
        'a=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id\r\n' +
        'a=recvonly\r\n' +
        'a=rtcp-mux\r\n' +
        'a=rtcp-rsize\r\n' +
        'a=rtpmap:123 AV1/90000\r\n' +
        'a=rtcp-fb:123 goog-remb\r\n' +
        'a=rtcp-fb:123 transport-cc\r\n' +
        'a=rtcp-fb:123 ccm fir\r\n' +
        'a=rtcp-fb:123 nack\r\n' +
        'a=rtcp-fb:123 nack pli\r\n' +
        'a=rtpmap:114 red/90000\r\n' +
        'a=rtpmap:115 rtx/90000\r\n' +
        'a=fmtp:115 apt=114\r\n' +
        'a=rtpmap:116 ulpfec/90000\r\n' +
        'a=rtpmap:119 rtx/90000\r\n' +
        'a=fmtp:119 apt=123\r\n',
      type: 'offer'
    }
  ],
  corrID: 39,
  replyTo: 'amq.gen-WtoELIbC4gJ1GfdYgkvSFA'
}

2. WebrtcNode - onTransportSignaling

dist/webrtc_agent/webrtc/index.js

// connectionId 应该就是operationId
that.onTransportSignaling = function (connectionId, msg, callback) {
        log.debug('onTransportSignaling, connection id:', connectionId, 'msg:', msg);
        // 通过 operationId 从mappingTransports 获取 transportId,
        // 接着通过transportId从peerConnections 获取WrtcConnection
        // 参考小结 2.1
        var conn = getWebRTCConnection(connectionId);
        // WrtcConnection conn
        if (conn) {
            // 参考小结 2.2
            conn.onSignalling(msg, connectionId);
            callback('callback', 'ok');
        } else {
          callback('callback', 'error', 'Connection does NOT exist:' + connectionId);
        }
    };

2.1 WebrtcNode - getWebRTCConnection——返回WrtcConnection

dist/webrtc_agent/webrtc/index.js

    // 返回的是WrtcConnection
    var getWebRTCConnection = function (operationId) {
        // 在WebrtcNode-createWebRTCConnection 创建
        var transportId = mappingTransports.get(operationId);
        if (peerConnections.has(transportId)) {
            return peerConnections.get(transportId);
        }
        return null;
    };

2.2 WrtcConnection.onSignalling

dist/webrtc_agent/webrtc/wrtcConnection.js

处理 offer sdp。

  that.onSignalling = function (msg, operationId) {
    var processSignalling = function () {
  // 这里type=’offer‘
      if (msg.type === 'offer') {
        log.debug('on offer:', msg.sdp);
        processOffer(msg.sdp);
      } else if (msg.type === 'candidate') {
      ...
      } else if (msg.type === 'removed-candidates') {
      ...
      }
    };
    // wrtc 是 Connection
    if (wrtc) {
      processSignalling();
    } else {
      // should not reach here
      log.error('wrtc is not ready');
    }
  };

2.3 ==========WrtcConnection.processOffer

dist/webrtc_agent/webrtc/wrtcConnection.js

这里的 remoteSdp 是null,

 const processOffer = function (sdp) {
 // 第一进来,remoteSdp 是null
    if (!remoteSdp) {
      // 1. First offer
      remoteSdp = new SdpInfo(sdp);
      // Check mid, 获取到mid 数组
      const mids = remoteSdp.mids();
      for (const mid of mids) {
         // 2. 设置finalformat
        processOfferMedia(mid);
      }
      // 3. 创建answer sdp
      localSdp = remoteSdp.answer();

      // 4. Setup transport
      let opId = null;
      for (const mid of mids) {
        if (remoteSdp.getMediaPort(mid) !== 0) {
          opId = setupTransport(mid);
        }
      }
      if (opId) {
        on_track({
          type: 'tracks-complete',
          operationId: opId
        });
      }

    } else {
     ...
    }
  };
  var remoteSdp = null;
  var localSdp = null;

2.3.1 SdpInfo.SdpInfo

dist/webrtc_agent/webrtc/sdpInfo.js

根据字符串,生成SdpInfo对象。

2.3.2 WrtcConnection.processOfferMedia——设置finalformat

WrtcConnection.addTrackOperation, 创建了元素operationMap

设置finalFormat

 const processOfferMedia = function (mid) {
    ...

    // Determine media format in offer
    if (remoteSdp.mediaType(mid) === 'audio') {
      const audioPreference = operationMap.get(mid).formatPreference;
      // filter audio payload
      const audioFormat = remoteSdp.filterAudioPayload(mid, audioPreference);
      operationMap.get(mid).finalFormat = audioFormat;
    } else if (remoteSdp.mediaType(mid) === 'video') {
      // 这是publish的存储的数据
      const videoPreference = operationMap.get(mid).formatPreference;
      // filter video payload    
      const videoFormat = remoteSdp.filterVideoPayload(mid, videoPreference);
      // 设置 finalFormat
      operationMap.get(mid).finalFormat = videoFormat;
    }
  };

operationMap 的元素就是WrtcConnection.addTrackOperation中添加的,这里是给finalFormat赋值

// mid => { operationId, sdpDirection, type, formatPreference, rids, enabled, finalFormat }
  var operationMap = new Map();

dist/webrtc_agent/webrtc/wrtcConnection.js,在WebrtcNode-subscirbe.md 的addTrackOperation 小节 有详细说明。

SdpInfo.filterVideoPayload

2023-05-31T16:44:11.144  - DEBUG: SdpInfo - filterVideoPayload,
mid=1,
preferenece={"optional":[{"codec":"h264"},
{"codec":"vp8"},{"codec":"vp9"},
{"codec":"av1"},{"codec":"h265"}]}

2023-05-31T16:44:11.145  - DEBUG: SdpInfo - filterVideoPayload, 
finalFmt video: {"codec":"av1"}

mediaInfo.rtp

rtp:[
                {
                    "payload":123,
                    "codec":"AV1",
                    "rate":90000
                },
                {
                    "payload":114,
                    "codec":"red",
                    "rate":90000
                },
                {
                    "payload":115,
                    "codec":"rtx",
                    "rate":90000
                },
                {
                    "payload":116,
                    "codec":"ulpfec",
                    "rate":90000
                },
                {
                    "payload":119,
                    "codec":"rtx",
                    "rate":90000
                }
            ],
filterVideoPayload(mid, preference) {
    log.debug("filterVideoPayload,mid="+ mid +",preferenece="+JSON.stringify(preference));
    // Remove unsupported profiles
    filterVP9(this.obj, preference, mid);
    const finalPrf = filterH264(this.obj, preference, mid);
    let finalFmt = null;
    let selectedPayload = -1;
    // 空
    const preferred = preference.preferred;
    const optionals = preference.optional || [];
    const relatedPayloads = new Set();
    const allowedFbTypes = [
      'ccm fir',
      'nack',
      'transport-cc',
      'goog-remb',
    ];
    const reservedCodecs = ['red', 'ulpfec'];
    const codecMap = new Map();
    const payloadOrder = new Map();

    const mediaInfo = this.media(mid);
    if (mediaInfo && mediaInfo.type == 'video') {
      let rtp, fmtp;
      let payloads;
      // Keep payload order in m line
      mediaInfo.payloads.toString().split(' ')
        .forEach((p, index) => {
          payloadOrder.set(parseInt(p), index);
        });
      if (mediaInfo.direction === 'recvonly') {
        // For subscription
        // concat(optionals.map((fmt) => fmt.codec.toLowerCase()));
        optionals.forEach((fmt) => {
          reservedCodecs.push(fmt.codec.toLowerCase());
        });
      }

      // mediaInfo.rtp 就是上面sdp里的payload
      for (let i = 0; i < mediaInfo.rtp.length; i++) {
        rtp = mediaInfo.rtp[i];
        const codec = rtp.codec.toLowerCase();
        if (reservedCodecs.includes(codec)) {
          relatedPayloads.add(rtp.payload);
        }
        codecMap.set(rtp.payload, codec);
        // 这个条件不会进入
        if (preferred && preferred.codec === codec) {
          selectedPayload = rtp.payload;
          break;
        }
        // 在optional 中找mediaInfo.rtp 的codec
        if (optionals.findIndex((fmt) => (fmt.codec === codec)) > -1) {
          if (selectedPayload < 0 ||
              payloadOrder.get(rtp.payload) < payloadOrder.get(selectedPayload)) {
            selectedPayload = rtp.payload;
          }
        }
      }
      // TODO: uncomment following code after register rtx in video receiver
      // if (!mediaInfo.simulcast) {
      //   for (i = 0; i < mediaInfo.fmtp.length; i++) {
      //     fmtp = mediaInfo.fmtp[i];
      //     if (fmtp.config.indexOf(`apt=${selectedPayload}`) > -1) {
      //       relatedPayloads.add(fmtp.payload);
      //     }
      //   }
      // }

      // Remove ulpfec if h264/h265 is selected
      const selectedCodec = codecMap.get(selectedPayload);
      if (['h264', 'h265'].includes(selectedCodec)) {
        codecMap.forEach((codec, pt) => {
          if (codec === 'ulpfec') {
            relatedPayloads.delete(pt);
          }
        });
      }
      // Remove red if vp9 SVC
      if (selectedCodec === 'vp9' && this.getLegacySimulcast(mid)) {
        codecMap.forEach((codec, pt) => {
          if (codec === 'red') {
            relatedPayloads.delete(pt);
          }
        });
      }

      relatedPayloads.add(selectedPayload);
      // Remove non-selected video payload
      mediaInfo.rtp = mediaInfo.rtp.filter(
        (rtp) => relatedPayloads.has(rtp.payload));
      if (mediaInfo.fmtp) {
        mediaInfo.fmtp = mediaInfo.fmtp.filter(
          (fmtp) => relatedPayloads.has(fmtp.payload));
      }
      if (mediaInfo.rtcpFb) {
        mediaInfo.rtcpFb = mediaInfo.rtcpFb.filter(
          (rtcp) => allowedFbTypes.includes(rtcp.type));
        mediaInfo.rtcpFb = mediaInfo.rtcpFb.filter(
          (rtcp) => relatedPayloads.has(rtcp.payload));
      }
      const payloadList = mediaInfo.payloads.toString().split(' ');
      if (selectedPayload !== -1) {
        payloadList.unshift(selectedPayload.toString());
      }
      mediaInfo.payloads = payloadList
        .filter((p) => relatedPayloads.has(parseInt(p)))
        .filter((v, index, self) => self.indexOf(v) === index)
        .join(' ');
    }

    if (selectedPayload !== -1) {
      finalFmt = { codec: codecMap.get(selectedPayload) };
      if (finalFmt.codec === 'h264' && finalPrf) {
        finalFmt.profile = finalPrf;
      }
    }
    log.debug('filterVideoPayload, finalFmt video:', JSON.stringify(finalFmt));
    return finalFmt;
  }

2.3.3 SdpInfo.answer——localSdp

通过offer到sdp,进行对应的修改,作为answer的sdp。赋值给localSdp中,在WrtcConnection.setupTransport用到。

 answer() {
    const answer = new SdpInfo(this.toString());
    answer.obj.origin = {
      username: '-',
      sessionId: '0',
      sessionVersion: 0,
      netType: 'IN',
      ipVer: 4,
      address: '127.0.0.1'
    };
    answer.obj.media.forEach(mediaInfo => {
      mediaInfo.port = 1;
      mediaInfo.rtcp = {
        port: 1,
        netType: 'IN',
        ipVer: 4,
        address: '0.0.0.0'
      };
      if (mediaInfo.setup === 'active') {
        mediaInfo.setup = 'passive';
      } else {
        mediaInfo.setup = 'active';
      }

      delete mediaInfo.iceOptions;  
      delete mediaInfo.rtcpRsize;
      if (mediaInfo.direction === 'recvonly') {
        mediaInfo.direction = 'sendonly';
      } else if (mediaInfo.direction === 'sendonly') {
        delete mediaInfo.msid;
        delete mediaInfo.ssrcGroups;
        delete mediaInfo.ssrcs;
        mediaInfo.direction = 'recvonly';
      }

      if (mediaInfo.ext && Array.isArray(mediaInfo.ext)) {
        const extMappings = [
          'urn:ietf:params:rtp-hdrext:ssrc-audio-level',
          // 'http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01',
          'urn:ietf:params:rtp-hdrext:sdes:mid',
          'urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id',
          'urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id',
          'urn:ietf:params:rtp-hdrext:toffset',
          'http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time',
          // 'urn:3gpp:video-orientation',
          // 'http://www.webrtc.org/experiments/rtp-hdrext/playout-delay',
        ];
        if (mediaInfo.type === 'video') {
          extMappings.push(
            'http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01'
          );
        }
        mediaInfo.ext = mediaInfo.ext.filter((e) => {
          return extMappings.includes(e.uri);
        });
      }

      if (mediaInfo.rids && Array.isArray(mediaInfo.rids)) {
        // Reverse rids direction
        mediaInfo.rids.forEach(r => {
          r.direction = (r.direction === 'send') ? 'recv' : 'send';
        });
      }
      if (mediaInfo.simulcast) {
        // Reverse simulcast direction
        if (mediaInfo.simulcast.dir1 === 'send') {
          mediaInfo.simulcast.dir1 = 'recv';
        } else {
          mediaInfo.simulcast.dir1 = 'send';
        }
      }
    });

    return answer;
  }

SdpInfo.getMediaPort

m=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 102 0 8 106 105 13 110 112 113 126

m=video 9 UDP/TLS/RTP/SAVPF 123 114 115 116 119

9 就是port

SdpInfo.mediaType

m=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 102 0 8 106 105 13 110 112 113 126

m=video 9 UDP/TLS/RTP/SAVPF 123 114 115 116 119

auido/video, 就是mediaType

operationMap——存放的是track相关的内容

dist/webrtc_agent/webrtc/wrtcConnection.js

WebrtcNode-subscribe.md 的addTrackOperation 小节 有详细说明

  // mid => { operationId, sdpDirection, type, formatPreference, rids, enabled, finalFormat }
  var operationMap = new Map();

2.3.4 WrtcConnection.setupTransport——创建WrtcStream

dist/webrtc_agent/webrtc/wrtcConnection.js

小节 3

2.3.5 on_track——这是创建WrtcConnection 传入的callback,tracks-complete

WebrtcNode-subscribe.md3.WebrtcNode - createWebRTCConnection——返回WrtcConnection 中传入了

function onTrack(trackInfo) {
            handleTrackInfo(transportId, trackInfo, controller);
        }

调用的代码

        on_track({
          type: 'tracks-complete',
          operationId: opId
        });

详细的看3.3, 3.4 小节

3. WrtcConnection.setupTransport——创建WrtcStream

主要是创建MediaStream。

dist/webrtc_agent/webrtc/wrtcConnection.js

2023-04-26T21:54:19.800  - DEBUG: WrtcConnection - Add ssrc 1735005623 
to video in SDP for b149e44bb10d4e91bd162a8c6806ae7b

2023-04-26T21:54:19.800  - DEBUG: SdpInfo - Set SSRC: 1 [
{"id":1735005623,"attribute":"cname","value":"o/i14u9pJrxRKAsu"},
{"id":1735005623,"attribute":"msid","value":"guSQ9WN0ZG v0"},
{"id":1735005623,"attribute":"mslabel","value":"guSQ9WN0ZG"},
{"id":1735005623,"attribute":"label","value":"guSQ9WN0ZGv0"}]
2023-04-26 21:54:19,802  - DEBUG: WebRtcConnection - 
id: b149e44bb10d4e91bd162a8c6806ae7b,  message: setting remote SDP
2023-04-26 21:54:19,802  - DEBUG: WebRtcConnection - 
id: b149e44bb10d4e91bd162a8c6806ae7b,  message: processing remote SDP
const setupTransport = function (mid) {
    let rids = remoteSdp.rids(mid);
    const opSettings = operationMap.get(mid);
      // recvonly, 是out
    const direction = (opSettings.sdpDirection === 'sendonly') ? 'in' : 'out';
    const simSsrcs = remoteSdp.getLegacySimulcast(mid);
    const trackSettings = remoteSdp.getMediaSettings(mid);
    const mediaType = remoteSdp.mediaType(mid);

    trackSettings.owner = owner; // 用户id
    trackSettings.enableBWE = that.enableBWE;
    if (opSettings.finalFormat) {
      trackSettings[mediaType].format = opSettings.finalFormat;
      if (opSettings.finalFormat.codec === 'vp9' && simSsrcs) {
        // Legacy simulcast for VP9 SVC
        rids = null;
        trackSettings['video'].scalabilityMode = opSettings.scalabilityMode;
      }
    }

    if (rids) {
        ...
    } else {
        // 走这里的代码
      // No simulcast
      if (!trackMap.has(mid)) {
        // 1. Connection wrtc
        trackMap.set(mid, new WrtcStream(mid, wrtc, direction, trackSettings));
        // Set ssrc in local sdp for out direction
        // direct 是out
        if (direction === 'out') {
          // mtype = ‘video’
          const mtype = localSdp.mediaType(mid);
          // 2. 在WrtcStream 创建的时候,创建了videoFramePacketizer
          // 从WrtcStream 获取ssrc
          const ssrc = trackMap.get(mid).ssrc(mtype);
          if (ssrc) {
            log.debug(`Add ssrc ${ssrc} to ${mtype} in SDP for ${wrtcId}`);
            const opId = opSettings.operationId;
            let msid = msidMap.get(opId);
            // 3. setSsrcs
            if (msid) {
              localSdp.setSsrcs(mid, [ssrc], msid);
            } else {
              msid = localSdp.setSsrcs(mid, [ssrc]);
              msidMap.set(opId, msid);
            }
          }
        }
        // 4. Connection wrtc
        // remoteSdp 是在processOffer 创建的对象, client端发过来的sdp
        wrtc.setRemoteSdp(remoteSdp.singleMediaSdp(mid).toString(), mid);
        // 5. Notify new track, 详细见小节 3.5, 3.6
        // on_track, wrtcConnection 就是在创建的时候,传入的参数
        on_track({
          type: 'track-added',
          track: trackMap.get(mid),// WrtcStream
          operationId: opSettings.operationId,
          mid: mid
        });
      } else {
        log.warn(`Conflict trackId ${mid} for ${wrtcId}`);
      }
    }

    return opSettings.operationId;
  };

SdpInfo.rids

trackMap 属性——存放了WrtcStream

  // composedId => WrtcStream
  var trackMap = new Map();

composeId

  const composeId = function (mid, rid) {
    return mid + ':' + rid;
  };

3.1 new WrtcStream

小节 4
创建了MediaStream

见文章 WebrtcNode-subscribe-sdpOffer(2).md

>>>>>>>>>>>>不同于publish的流程>>>>>>>>>>

3.2 WrtcStream.ssrc

dist/webrtc_agent/webrtc/wrtcConnection.js

  ssrc(track) {
    if (track === 'audio' && this.audioFramePacketizer) {
      return this.audioFramePacketizer.ssrc();
    }
    if (track === 'video' && this.videoFramePacketizer) {
      return this.videoFramePacketizer.ssrc();
    }
    return null;
  }

3.2.1 addon.VideoFramePacketizer::getSsrc

source/agent/webrtc/rtcFrame/VideoFramePacketizerWrapper.cc

void VideoFramePacketizer::getSsrc(const v8::FunctionCallbackInfo<v8::Value>& args) {
  Isolate* isolate = Isolate::GetCurrent();
  HandleScope scope(isolate);

  VideoFramePacketizer* obj = ObjectWrap::Unwrap<VideoFramePacketizer>(args.Holder());
  owt_base::VideoFramePacketizer* me = obj->me;

  uint32_t ssrc = me->getSsrc();
  args.GetReturnValue().Set(Number::New(isolate, ssrc));
}

3.2.2 owt_base::VideoFramePacketizer

    uint32_t m_ssrc;
    uint32_t getSsrc() { return m_ssrc; }
bool VideoFramePacketizer::init(VideoFramePacketizer::Config& config)
{
    if (!m_videoSend) {
        // Create Send Video Stream
        ...
        m_videoSend = m_rtcAdapter->createVideoSender(sendConfig);
        m_ssrc = m_videoSend->ssrc();
        return true;
    }

    return false;
}

3.3 ???SdpInfo.setSsrcs

这里设置sscrs是什么作用

// ssrcs = [1735005623]
setSsrcs(mid, ssrcs, msid) {
    log.debug('setSsrcs,mid='+mid+",ssrcs="+ssrcs+",msid="+msid);
    const media = this.media(mid);
    if (!media) {
      return null;
    }
    if (!msid) {
      // Generate msid
      const alphanum = '0123456789' +
        'ABCDEFGHIJKLMNOPQRSTUVWXYZ' +
        'abcdefghijklmnopqrstuvwxyz';
      const msidLength = 10;
      msid = '';
      for (let i = 0; i < msidLength; i++) {
        msid += alphanum[Math.floor(Math.random() * alphanum.length)];
      }
    }

    const mtype = (media.type === 'audio') ? 'a' : 'v';
    // Only support one ssrc now
    const ssrc = ssrcs[0];
    media.ssrcs = [
      {id: ssrc, attribute: 'cname', value: 'o/i14u9pJrxRKAsu'},
      {id: ssrc, attribute: 'msid', value: `${msid} ${mtype}0`},
      {id: ssrc, attribute: 'mslabel', value: msid},
      {id: ssrc, attribute: 'label', value: `${msid}${mtype}0`},
    ];
    log.debug('Set SSRC:', mid, msid, JSON.stringify(media.ssrcs));
    return msid;
  }

<<<<<<<<<<<<<<<<<不同于publish的流程<<<<<<<<<<

3.4 Connection.setRemoteSdp

// streamId = mid
setRemoteSdp(sdp, streamId) {
    // webRtcConnection wrtc
    this.wrtc.setRemoteSdp(sdp, streamId || this.id);
  }

3.4.1 NAN_METHOD(WebRtcConnection::setRemoteSdp)

source/agent/webrtc/rtcConn/WebRtcConnection.cc

NAN_METHOD(WebRtcConnection::setRemoteSdp) {
  WebRtcConnection* obj = Nan::ObjectWrap::Unwrap<WebRtcConnection>(info.Holder());
  std::shared_ptr<erizo::WebRtcConnection> me = obj->me;
  if (!me) {
    return;
  }

  std::string sdp = getString(info[0]);
  std::string stream_id = getString(info[1]);

  bool r = me->setRemoteSdp(sdp, stream_id);

  info.GetReturnValue().Set(Nan::New(r));
}

3.4.2 erizo::WebRtcConnection::setRemoteSdp

source/agent/webrtc/rtcConn/erizo/src/erizo/WebRtcConnection.cpp

2023-04-26 21:54:19,802  - DEBUG: WebRtcConnection - 
id: b149e44bb10d4e91bd162a8c6806ae7b, 
 message: setting remote SDP
bool WebRtcConnection::setRemoteSdpInfo(std::shared_ptr<SdpInfo> sdp, std::string stream_id) {
  asyncTask([sdp, stream_id] (std::shared_ptr<WebRtcConnection> connection) {
    ELOG_DEBUG("%s message: setting remote SDPInfo", connection->toLog());

    if (!connection->sending_) {
      return;
    }

    connection->remote_sdp_ = sdp;
    // mid 
    connection->processRemoteSdp(stream_id);
  });
  return true;
}

??? erizo::WebRtcConnection::processRemoteSdp

source/agent/webrtc/rtcConn/erizo/src/erizo/WebRtcConnection.cpp

2023-04-26 21:54:19,802  - DEBUG: WebRtcConnection - id: b149e44bb10d4e91bd162a8c6806ae7b,  message: processing remote SDP

见后文owt-server/WebRTCEvent.md

SdpInfo.singleMediaSdp

dist-debug/webrtc_agent/webrtc/sdpInfo.js

  singleMediaSdp(mid) {
    const sdp = new SdpInfo(this.toString());
    sdp.obj.media = sdp.obj.media.filter(m => m.mid.toString() === mid);
    sdp.setBundleMids([mid]);
    return sdp;
  }

3.5 on__track——track add

dist/webrtc_agent/webrtc/wrtcConnection.js

通知到WebrtcNode(/dist/webrtc_agent/webrtc/index.js)的createWebRTCConnection 中注册的callback, 回调一个对象,

        // Notify new track
        on_track({
          type: 'track-added',
          track: trackMap.get(mid), // WrtcStream
          operationId: opSettings.operationId,
          mid: mid
        })

3.6 ======WebrtcNode.handleTrackInfo

dist/webrtc_agent/webrtc/index.js

// 这里传入onTrack, 就是on_track,就是 callback
var connection = new WrtcConnection({
            connectionId: transportId,
            threadPool: threadPool,
            ioThreadPool: ioThreadPool,
            network_interfaces: global.config.webrtc.network_interfaces,
            owner,
        }, function onTransportStatus(status) {
            notifyTransportStatus(controller, transportId, status);
        }, function onTrack(trackInfo) {
            /* trackInfo 就是 {
              type: 'track-added',
              track: trackMap.get(mid), // WrtcStream
              operationId: opSettings.operationId,
              mid: mid
            }*/
            handleTrackInfo(transportId, trackInfo, controller);
        });
// trackInfo 就是on__track 回调回来
var handleTrackInfo = function (transportId, trackInfo, controller) {
        var publicTrackId;
        var updateInfo;
        if (trackInfo.type === 'track-added') {
            // Generate public track ID
            const track = trackInfo.track; // WrtcStream
            publicTrackId = transportId + '-' + track.id;
            if (mediaTracks.has(publicTrackId)) {
                log.error('Conflict public track id:', publicTrackId, transportId, track.id);
                return;
            }
            mediaTracks.set(publicTrackId, track);
            mappingPublicId.get(transportId).set(track.id, publicTrackId);
            if (track.direction === 'in') {
                const trackSource = track.sender();
                router.addLocalSource(publicTrackId, 'webrtc', trackSource)
                .catch(e => log.warn('Unexpected error during track add:', e));
            } else {
                // 走这里
                router.addLocalDestination(publicTrackId, 'webrtc', track)
                .catch(e => log.warn('Unexpected error during track add:', e));
            }

            // Bind media-update handler
            track.on('media-update', (jsonUpdate) => {
                log.debug('notifyMediaUpdate:', publicTrackId, jsonUpdate);
                notifyMediaUpdate(controller, publicTrackId, track.direction, JSON.parse(jsonUpdate));
            });
            // Notify controller
            const mediaType = track.format('audio') ? 'audio' : 'video';
            updateInfo = {
                type: 'track-added',
                trackId: publicTrackId,
                mediaType: track.format('audio') ? 'audio' : 'video',
                mediaFormat: track.format(mediaType),
                direction: track.direction,
                operationId: trackInfo.operationId,
                mid: trackInfo.mid,
                rid: trackInfo.rid,
                active: true,
            };
            log.debug('notifyTrackUpdate', controller, publicTrackId, updateInfo);
            notifyTrackUpdate(controller, transportId, updateInfo);

        } else if (trackInfo.type === 'track-removed') {
            publicTrackId = mappingPublicId.get(transportId).get(trackInfo.trackId);
            if (!mediaTracks.has(publicTrackId)) {
                log.error('Non-exist public track id:', publicTrackId, transportId, trackInfo.trackId);
                return;
            }
            log.debug('track removed:', publicTrackId);
            router.removeConnection(publicTrackId)
            .then(ok => {
                mediaTracks.get(publicTrackId).close();
                mediaTracks.delete(publicTrackId);
                mappingPublicId.get(transportId).delete(trackInfo.trackId);
            })
            .catch(e => log.warn('Unexpected error during track remove:', e));

            // Notify controller
            updateInfo = {
                type: 'track-removed',
                trackId: publicTrackId,
            };
            notifyTrackUpdate(controller, transportId, updateInfo);

        } else if (trackInfo.type === 'tracks-complete') {
            updateInfo = {
                type: 'tracks-complete',
                operationId: trackInfo.operationId
            };
            notifyTrackUpdate(controller, transportId, updateInfo);
        }
    };

详细说明见后文 WebrtcNode-subscribe-sdpOffer(3).md2. WebrtcNode.handleTrackInfo

流程图

在这里插入图片描述

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:/a/25704.html

如若内容造成侵权/违法违规/事实不符,请联系我们进行投诉反馈qq邮箱809451989@qq.com,一经查实,立即删除!

相关文章

生活-考驾照2

昨日已通过科目一&#xff0c;接下来&#xff0c;向着科目二出发&#xff01;&#xff01;&#xff01; 考试注意事项 就考前一周死命刷题&#xff0c;至少一天有三次93以上的记录&#xff0c;我也就最后一天达到90&#xff0c;之前一直马路杀手刷题&#xff0c;我就在网页版…

WRF模式应用:天气预报、模拟分析观测气温、降水、风场、水汽和湿度、土地利用变化、土壤及近地层能量水分通量、土壤、水体、植被等相关气象变量

查看原文>>>高精度气象模拟软件WRF(Weather Research Forecasting)技术及案例应用 气候是多个领域&#xff08;生态、水资源、风资源及碳中和等问题&#xff09;的主要驱动因素&#xff0c;合理认知气候变化有利于解释生态环境变化机理及过程&#xff0c;而了解现在、…

Alibaba Arthas学习与使用

Alibaba Arthas学习与使用 目录 下载安装卸载退出快捷键重点部分: 命令 dashboardthreadjvmsyspropsysenvvmoptiongetstaticognlscsmjadmcredefinedumpclassloadermonitorwatchtracestackttoptionsprofiler 下载安装 # 下载 curl -O https://alibaba.github.io/arthas/art…

如何在多个端口上运行 SSH 服务器?

SSH&#xff08;Secure Shell&#xff09;是一种用于安全远程访问和管理服务器的协议。默认情况下&#xff0c;SSH服务器在Linux系统上使用22号端口进行通信。但是&#xff0c;有时我们可能需要在多个端口上运行SSH服务器&#xff0c;以满足特定的需求或增强服务器的安全性。 本…

SQL审核工具Yearning搭建及使用

一、背景 实际的业务场景中&#xff0c;我们难免会直接对生产库的数据进行修改。很多朋友都是发sql给运维&#xff0c;然后运维执行。过程不可控&#xff0c;出错几率很大。 而且没有审核&#xff0c;不知道提交的具体是什么sql&#xff0c;极有可能是删库跑路的sql&#xff0…

实验二十一、积分运算电路的输出波形分析

一、题目 利用 Multisim 分析图1所示两个积分运算电路的输出波形&#xff0c;输入电压为 200 Hz、幅值为 1 V 的方波信号。 图 1 图1\,\, 图1 二、仿真电路 在 Multism 中搭建图1所示的两个电路&#xff0c;如图2所示。为了防止电路中的直流增益过大&#xff0c;故在电容上…

【Python FTP/SFTP】零基础也能轻松掌握的学习路线与参考资料

一、Python FTP/SFTP的学习路线 Python FTP/SFTP是Python语言的两种常用的文件传输协议。在学习Python网络编程过程中&#xff0c;学习FTP/SFTP是非常重要的一步。下面给出Python FTP/SFTP的学习路线&#xff1a; 了解FTP/SFTP协议 在开始学习Python FTP/SFTP之前&#xff0…

【六一】【海思SS528】GPIO寄存器操作 - 使能GPIO管脚输出高、低电平

目录 一、概述二、配置复用控制寄存器&#xff0c;使能GPIO功能三、配置GPIO_DIR寄存器&#xff0c;选择输出四、配置GPIO_DATA寄存器&#xff0c;输出高电平五、测试 一、概述 这篇文章根据海思SS528芯片提供的《22AP30 H.265编解码处理器用户指南.pdf》文档(文档路径&#xf…

TPO69 01|Why Snakes Have Forked Tongues P5P6|阅读真题精读|17:50~19:35

17:00&#xff5e;17:50 吃饭 目录 P56 生词 段落大意 P5段落大意 P6段落大意 题目 【5】事实信息题|定位准确非常重要✅ 【6】事实信息题|定位准确非常重要✅ 【7】推理题|文章是否提及|不要过度推理 【8】修辞目的题|举例一般为了说明✅ 【9】句子插入题|in other words|同义…

读数据压缩入门笔记01_数据压缩导读

1. 建立在数据压缩上的世界 1.1. 数据压缩技术最让人惊异之处在于&#xff0c;它与过去40年里个人计算的很多重大改变有关&#xff0c;但很少有人知道这一点 1.2. 我们当下生活在其中的这个计算世界&#xff0c;完全建立在数据压缩算法之上 1.3. 图像的压缩 1.3.1. 1978年 …

Matcher: Segment Anything with One Shot Using All-Purpose Feature Matching 论文精读

Matcher: Segment Anything with One Shot Using All-Purpose Feature Matching 论文链接&#xff1a;[2305.13310] Matcher: Segment Anything with One Shot Using All-Purpose Feature Matching (arxiv.org) 代码链接&#xff1a;aim-uofa/Matcher: Matcher: Segment Anyt…

图的邻接矩阵表示

设图有n个顶点&#xff0c;则邻接矩阵是一个n*n的方阵&#xff1b;若2个顶点之间有边&#xff0c;则方阵对应位置的值为1&#xff0c;否则为0&#xff1b; 看几个例子&#xff1b; 此图的邻接矩阵是 0 1 1 1 1 0 1 0 1 1 0 1 1 0…

计算机网络基础知识(七)—— 什么是HTTPS协议?你听我“瞎掰”

文章目录 01 | 工作原理02 | SSL/TLS协议2.1 | 握手协议2.2 | 更换密码协议&#xff08;Change Cipher Spec Protocol&#xff09;2.3 | 警告协议&#xff08;Alert Protocol&#xff09;2.4 | 应用数据协议&#xff08;Application Data Protocol&#xff09; 03 | 加密算法3.…

南大通用数据库-Gbase-8a-报错集锦-02-metadata is incomplete on localhost

一、版本信息 名称值CPUIntel(R) Core(TM) i5-1035G1 CPU 1.00GHz操作系统CentOS Linux release 7.9.2009 (Core)内存3G逻辑核数2Gbase8a版本8.6.2-R43 二、问题原因 由于gbase.table_distribution存储了所有引擎为express的表元数据信息&#xff0c;如果此表出现数据损坏&a…

【五】设计模式~~~创建型模式~~~单例模式(Java)

【学习难度&#xff1a;★☆☆☆☆&#xff0c;使用频率&#xff1a;★★★★☆】 5.1. 模式动机 对于系统中的某些类来说&#xff0c;只有一个实例很重要&#xff0c;例如&#xff0c;一个系统中可以存在多个打印任务&#xff0c;但是只能有一个正在工作的任务&#xff1b;一…

Hystrix 服务熔断

书籍,人,借阅服务之间相互调用, 高度耦合, 一旦一个服务故障, 其他服务会雪崩, 和多米诺骨牌一样 Hystrix 熔断器, 保险丝 服务降级 提供补救措施发给请求者, 服务可用, 能力下降了 borrow-service 导入依赖 <dependency><groupId>org.springframework.cloud&l…

【MySQL新手到通关】第六章 时间日期函数

文章目录 1.获取日期时间函数1.1 获取当前日期时间1.2 获取当前日期1.3 获取当前时间 2.日期格式化★★★2.1 日期转指定格式字符串2.2 字符串转日期 3.日期间隔3.1 增加日期间隔 ★★★3.2 减去一个时间间隔★★★3.3 日期相差天数&#xff08;天&#xff09;3.4 相差时间&…

使用Chat gpt提高Android开发效率

简介 在过去几周里&#xff0c;我进行了一项令人大开眼界的实验&#xff0c;将 Chat-GPT&#xff08;我使用的是 Bing Chat&#xff0c;它在后台使用了 GPT-4&#xff0c;并且可以免费使用&#xff09;融入到我的日常 Android 开发工作流程中&#xff0c;以探索它是否能够提高…

DAY04_JDBC快速入门JDBC API详解SQL防注入数据库连接池JDBC综合练习

目录 1 JDBC1.1 JDBC概念1.2 JDBC本质 1.3 JDBC好处 2 JDBC快速入门2.1 编写代码步骤2.2 具体操作 3 JDBC API详解3.1 DriverManager3.2 Connection3.2.1 获取执行对象3.2.2 事务管理 3.3 Statement3.4 ResultSet3.4.1 ResultSet案例 3.5 PreparedStatement3.5.1 SQL注入3.5.2 …

hutool文件导出

hutool文件导出 需求&#xff1a;管理员设置会议&#xff0c;参加会议会根据管理员设置的会议要求&#xff0c;用户参加会议填写相关数据&#xff0c;并且生成一个动态的excel数据并导出 示例&#xff1a; 每场都可以自定义报名字段 根据需求与前端约定 字段名称&#xff08;n…