Skip to content

web-client media path

В web-client media path построен вокруг общего MediaMuxSocket.

flowchart TB
    VG[VideograceClient.ts]
    Mux[media_mux_socket.js]
    MC[media_channel.js<br/>remote audio/video]
    Mic[mic_session.js<br/>local audio]
    Cam[cam_session.js<br/>local video]
    Screen[screen_session.js<br/>local screen]
    RTP[rtp_wsm_utils.js<br/>build/parse WSM frames]

    VG --> MC
    VG --> Mic
    VG --> Cam
    VG --> Screen
    MC --> Mux
    Mic --> Mux
    Cam --> Mux
    Screen --> Mux
    Mux --> RTP

Responsibilities

Module Responsibility
VideograceClient.ts Control lifecycle, conference state, device creation/deletion, UI integration.
media_mux_socket.js One WSMedia WebSocket, connect, reconnect, register/unregister handlers by ssrc, demux inbound frames.
media_channel.js Remote audio/video rendering, decoder lifecycle, RTP init, ForceKeyFrame.
mic_session.js Local microphone capture, Opus encode, RTP packetization, send through mux.
cam_session.js Local camera capture, VP8 encode, RTP split, RTCP keyframe handling.
screen_session.js Local screen capture, VP8 encode, RTP split.
rtp_wsm_utils.js WSM binary header build/parse, RTP helpers, AES-GCM payload crypto helpers.

Current browser invariant

flowchart LR
    One[One MediaMuxSocket]
    Many[Many SSRC handlers]
    Device[Device lifecycle via ControlWS]

    Device --> Many
    Many --> One

Browser media must not create per-device media WebSockets. If a new media type is added, it should register a new ssrc in MediaMuxSocket.

Local operation serialization

Fast mic/cam off/on operations are serialized at VideograceClient level. This is not a Safari-specific workaround; it protects device lifecycle from overlapping startLocalCapture, device_params, attachRemote, disconnect_device operations.

sequenceDiagram
    participant UI
    participant VG as VideograceClient
    participant Session as Mic/Camera Session
    participant Control as ControlWS
    participant Mux as MediaMuxSocket

    UI->>VG: toggleCam(true)
    VG->>Session: startLocalCapture()
    VG->>Control: device_params
    Control-->>VG: device_connect my=1
    VG->>Session: attachRemote(author_ssrc, port)
    Session->>Mux: register author_ssrc
    Session->>Mux: send RTP frames

Mux reconnect behavior

MediaMuxSocket owns reconnect for the shared media transport. On reconnect it calls registered handlers:

  • local media sessions mark _canEncode=true;
  • remote media channels send RTP init again;
  • remote video channels request a keyframe again.

Device creation is not repeated during media reconnect.