记录：利用 Agora 在 Unity3D MRTK场景中创建实时视频聊天应用

本质是两部带摄像机的设备同时进入Agora聊天室内视频。
去年实现过一次这个功能，用的是Agora_Unity_RTC_SDK 4.2.2版本的，今年使用失败，遂重新安装最新版本Agora_Unity_RTC_SDK（4.3.2）并按照官方教程配置，但发现不能正常使用。解决之后，特此记录。
本文假设已经有了Unity3D Scene并安装好了MRTK。

准备

名称	版本
设备A	使用其浏览器
设备B	安装了Unity
Unity	2202.3.9.f1
MRTK	2.8.3.0
Agora_Unity_RTC_SDK	4.3.2

建议使用两台设备测试来避免出现相机占用的问题。

1. 安装Agora_Unity_RTC_SDK

SDK下载链接：https://docs.agora.io/en/sdks?platform=unity，点击import就可以导入project中。

2. 创建UI

英文版官方说明：https://docs.agora.io/en/video-calling/get-started/get-started-sdk?platform=unity

我们需要（名字都不可以改动，会影响SDK的使用）：

名称	属性
TestCanvas	Canvas
LocalView	Raw Image
Join	Button
Leave	Button
RemoteView	Cube 我设的size:(1,1,0.01)

其中，RemoteView会放入MixedRealitySceneContent中以实现可以在AR场景中看见RemoteView中的内容；其他都放在创建的TestCanvas中。
在这里插入图片描述

3. script具体内容

我创建了JoinChannelView.cs并将其挂在上一步创建的TestCanvas上。以下是JoinChannelView.cs的具体内容：

using System;
using System.Linq;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.UI;
using UnityEngine.Serialization;
using Agora.Rtc;

#if (UNITY_2018_3_OR_NEWER && UNITY_ANDROID)
using UnityEngine.Android;
#endif

public class JoinChannelVideo : MonoBehaviour
{
    // Fill in your app ID
    private string _appID = "待填";
    // Fill in your channel name
    private string _channelName = "待填";
    // Fill in a temporary token
    private string _token = "待填";

    internal VideoSurface LocalView;
    internal VideoSurface RemoteView;
    internal IRtcEngine RtcEngine;
        
        
    #if (UNITY_2018_3_OR_NEWER && UNITY_ANDROID)
    private ArrayList permissionList = new ArrayList() { Permission.Camera, Permission.Microphone };
    #endif

    // Start is called before the first frame update
    void Start()
    {
        SetupVideoSDKEngine();
        InitEventHandler(); 
        SetupUI();
    }

    private void InitEventHandler()
    {
        UserEventHandler handler = new UserEventHandler(this);
        RtcEngine.InitEventHandler(handler);
    }

    private void SetupUI()
    {
        GameObject go  = GameObject.Find("RemoteView");
        RemoteView = go.AddComponent<VideoSurface>();
        go.transform.Rotate(0.0f, 0.0f, -180.0f);

        go = GameObject.Find("LocalView");
        LocalView = go.AddComponent<VideoSurface>();
        go.transform.Rotate(0.0f, 0.0f, -180.0f);

        go = GameObject.Find("Leave");
        go.GetComponent<Button>().onClick.AddListener(Leave);
        go = GameObject.Find("Join");
        go.GetComponent<Button>().onClick.AddListener(Join);

        //Join();
    }

    public void Join(){
        // Enable the video module
        RtcEngine.EnableVideo();
        // Set channel media options
        ChannelMediaOptions options = new ChannelMediaOptions();
        // Start video rendering
        LocalView.SetEnable(true);
        // Automatically subscribe to all audio streams
        options.autoSubscribeAudio.SetValue(true);
        // Automatically subscribe to all video streams
        options.autoSubscribeVideo.SetValue(true);
        // Set the channel profile to live broadcast
        options.channelProfile.SetValue(CHANNEL_PROFILE_TYPE.CHANNEL_PROFILE_COMMUNICATION);
        //Set the user role as host
        options.clientRoleType.SetValue(CLIENT_ROLE_TYPE.CLIENT_ROLE_BROADCASTER);
        // Join a channel
        RtcEngine.JoinChannel(_token, _channelName, 0, options);
    }
    public void Leave() 
    {
        Debug.Log("Leaving _channelName");
        // Leave the channel
        RtcEngine.LeaveChannel();
        // Disable the video module
        RtcEngine.DisableVideo();
        // Stop remote video rendering
        RemoteView.SetEnable(false);
        // Stop local video rendering
        LocalView.SetEnable(false);
    }

    // Update is called once per frame
    void Update() {
        CheckPermissions();
    }


    private void CheckPermissions() {
        #if (UNITY_2018_3_OR_NEWER && UNITY_ANDROID)
        foreach (string permission in permissionList) {
            if (!Permission.HasUserAuthorizedPermission(permission)) {
                Permission.RequestUserPermission(permission);
            }
        }
        #endif
    }

    private void SetupVideoSDKEngine()
    {
            // Create an IRtcEngine instance
            RtcEngine = Agora.Rtc.RtcEngine.CreateAgoraRtcEngine();
            RtcEngineContext context = new RtcEngineContext();
            context.appId = _appID;
            context.context = 0;
            context.channelProfile = CHANNEL_PROFILE_TYPE.CHANNEL_PROFILE_LIVE_BROADCASTING;
            context.audioScenario = AUDIO_SCENARIO_TYPE.AUDIO_SCENARIO_DEFAULT;
            // Initialize the instance
            RtcEngine.Initialize(context);
    }

}

// Implement your own EventHandler class by inheriting the IRtcEngineEventHandler interface class implementation
internal class UserEventHandler : IRtcEngineEventHandler
{
    private readonly JoinChannelVideo _videoSample;
    internal UserEventHandler(JoinChannelVideo videoSample)
    {
        _videoSample = videoSample;
    }
    // error callback
    public override void OnError(int err, string msg)
    {
    }
    // Triggered when a local user successfully joins the channel
    public override void OnJoinChannelSuccess(RtcConnection connection, int elapsed)
    {
        _videoSample.LocalView.SetForUser(0, "");
    }
    // When the SDK receives the first frame of a remote video stream and successfully decodes it, the OnUserJoined callback is triggered.
    public override void OnUserJoined(RtcConnection connection, uint uid, int elapsed)
    {
        // Set the remote video display
        _videoSample.RemoteView.SetForUser(uid, connection.channelId, VIDEO_SOURCE_TYPE.VIDEO_SOURCE_REMOTE);
        // Start video rendering
        _videoSample.RemoteView.SetEnable(true);
        Debug.Log("Remote user joined");
    }
}

这里需要注意的是函数SetupVideoSDKEngine，官方的写法是：

private void SetupVideoSDKEngine()
{
    // Create an IRtcEngine instance
    RtcEngine = Agora.Rtc.RtcEngine.CreateAgoraRtcEngine();
    RtcEngineContext context = new RtcEngineContext(_appID, 0,CHANNEL_PROFILE_TYPE.CHANNEL_PROFILE_LIVE_BROADCASTING, AUDIO_SCENARIO_TYPE.AUDIO_SCENARIO_DEFAULT);
    // Initialize the instance
    RtcEngine.Initialize(context);
}

感觉和我的修改的代码没有什么本质区别，但我测试过，用官方的教程无法获得RemoteView内容。

此外，internal class UserEventHandler : IRtcEngineEventHandler {}函数需要放在public class JoinChannelVideo : MonoBehaviour {}外面，之前由于对c#不熟悉将internal class UserEventHandler放在public class JoinChannelVideo里，虽然不会报错，但也不能正常使用。

4. 使用测试

注册Agora，并创建Project，注册连接：https://console.agora.io/v2。
在这里插入图片描述

把名为 My New Project 的 project 的 APP ID，Channel Name，Token 填入设备A打开的浏览器：Agora Basic Video Call 中连接：https://webdemo.agora.io/basicVideoCall/index.html#，点击 Join。
设备B 的Unity Project中，修改JoinChannelVideo内容：

public class JoinChannelVideo : MonoBehaviour
{
    // Fill in your app ID
    private string _appID = "e6fd32eba33741e68641f504580e7d29";
    // Fill in your channel name
    private string _channelName = "My New Project";
    // Fill in a temporary token
    private string _token = "007eJxTYHB7f9nyxhpbtncH1e8YsNm8qVX09fs7o6pqjvZ7A+u5wXoKDKlmaSnGRqlJicbG5iaGqWYWZiaGaaYGJqYWBqnmKUaW3zfkpjUEMjJwXz3LysgAgSA+H4NvpYJfarlCQFF+VmpyCQMDAIqPIqw=";

    internal VideoSurface LocalView;
    internal VideoSurface RemoteView;
    internal IRtcEngine RtcEngine;