Options
All
  • Public
  • Public/Protected
  • All
Menu

Class HiFiCommunicator

This class exposes properties and methods useful for communicating from the High Fidelity Audio API Client to the High Fidelity Audio API Server.

Hierarchy

  • HiFiCommunicator

Index

Constructors

constructor

  • Constructor for the HiFiCommunicator object. Once you have created a HiFiCommunicator, you can use the setInputAudioMediaStream method to assign an input audio stream to the connection, and once the connection has been established, use the getOutputAudioMediaStream method to retrieve the output audio from the server.

    Parameters

    • __namedParameters: { customSTUNandTURNConfig?: CustomSTUNandTURNConfig; hiFiAxisConfiguration?: HiFiAxisConfiguration; initialHiFiAudioAPIData?: HiFiAudioAPIData; onConnectionStateChanged?: Function; onMuteChanged?: OnMuteChangedCallback; onUsersDisconnected?: Function; transmitRateLimitTimeoutMS?: number; userDataStreamingScope?: HiFiUserDataStreamingScopes; webrtcSessionParams?: WebRTCSessionParams } = ...
      • Optional customSTUNandTURNConfig?: CustomSTUNandTURNConfig

        Cannot be set later. This object can be used if specific STUN and TURN server information needs to be provided for negotiating the underlying WebRTC connection. By default, High Fidelity's TURN server will be used, which should suffice for most operations. This is primarily useful for testing or for using a commercial TURN server provider for dealing with particularly challenging client networks/firewalls. See CustomSTUNandTURNConfig for the format of this object (note that all values must be provided when setting this).

      • Optional hiFiAxisConfiguration?: HiFiAxisConfiguration

        Cannot be set later. The 3D axis configuration. See ourHiFiAxisConfiguration for defaults.

      • Optional initialHiFiAudioAPIData?: HiFiAudioAPIData

        The initial position, orientation, etc of the user.

      • Optional onConnectionStateChanged?: Function

        A function that will be called when the connection state to the High Fidelity Audio API Server changes. See HiFiConnectionStates.

      • Optional onMuteChanged?: OnMuteChangedCallback

        A function that will be called when the mute state of the client has changed, for example when muted by an admin. See {@link OnMuteChangedCallback} for the information this function will receive.

      • Optional onUsersDisconnected?: Function

        A function that will be called when a peer disconnects from the Space.

      • Optional transmitRateLimitTimeoutMS?: number

        User Data updates will not be sent to the server any more frequently than this number in milliseconds.

      • Optional userDataStreamingScope?: HiFiUserDataStreamingScopes

        Cannot be set later. See HiFiUserDataStreamingScopes.

      • Optional webrtcSessionParams?: WebRTCSessionParams

        Cannot be set later. Extra parameters used for configuring the underlying WebRTC connection to the API servers. These settings are not frequently used; they are primarily for specific jitter buffer configurations.

    Returns HiFiCommunicator

Properties

Private _currentHiFiAudioAPIData

_currentHiFiAudioAPIData: HiFiAudioAPIData

Private Optional _customSTUNandTURNConfig

_customSTUNandTURNConfig: CustomSTUNandTURNConfig

Private _inputAudioMediaStream

_inputAudioMediaStream: MediaStream

Private _lastTransmittedHiFiAudioAPIData

_lastTransmittedHiFiAudioAPIData: HiFiAudioAPIData

Private _mixerSession

_mixerSession: HiFiMixerSession

Private _timers

_timers: any = ...

Private _userDataSubscriptions

_userDataSubscriptions: UserDataSubscription[]

Private Optional _webRTCSessionParams

_webRTCSessionParams: WebRTCSessionParams

onUsersDisconnected

onUsersDisconnected: Function

transmitRateLimitTimeoutMS

transmitRateLimitTimeoutMS: number

Methods

Private _cleanUpHiFiAudioAPIDataHistory

  • _cleanUpHiFiAudioAPIDataHistory(): void
  • Normally, we try to limit the amount of data we transmit to the High Fidelity Audio API server, by remembering what we sent. See _updateUserData for more information on how this is done.

    This function exists to handle any scenarios of remembering too much sent data. It is called just after data is succesfully sent, when data is known to no longer be needed.

    Returns void

Private _handleUserDataUpdates

  • Ingests user data updates from the server and, if relevant, calls the relevant callback functions associated with the User Data Subscriptions. See addUserDataSubscription.

    Parameters

    • newUserDataFromServer: ReceivedHiFiAudioAPIData[]

      Contains all of the new user data most recently received from the server.

    Returns void

Private _maybeClearRateLimitTimeout

  • _maybeClearRateLimitTimeout(): void
  • Clears the clientside rate limit timeout used to prevent user data from being sent to the High Fidelity Audio API server too often.

    Returns void

Private _onUsersDisconnected

  • A simple wrapper function called by our instantiation of HiFiMixerSession that calls the user-provided onUsersDisconnected() function if one exists. Library users can provide an onUsersDisconnected() callback function when instantiating the HiFiCommunicator object, or by setting HiFiCommunicator.onUsersDisconnected after instantiation.

    Parameters

    Returns void

Private _transmitHiFiAudioAPIDataToServer

  • _transmitHiFiAudioAPIDataToServer(forceTransmit?: boolean): any
  • Formats the local user data properly, then sends that user data to the High Fidelity Audio API server. This transfer is rate limited.

    There is no reason a library user would need to call this function without also simultaneously updating User Data, so this function is private.

    Parameters

    • Optional forceTransmit: boolean

      true if we should ignore the clientside rate limiter and send the data regardless of its status; false otherwise.

    Returns any

    If this operation is successful, returns { success: true, rawDataTransmitted: <the raw data that was transmitted to the server>}. If unsuccessful, returns { success: false, error: <an error message> }.

Private _updateLastTransmittedHiFiAudioAPIData

  • _updateLastTransmittedHiFiAudioAPIData(dataJustTransmitted: HiFiAudioAPIData): void
  • We keep a clientside copy of the data that we last transmitted to the High Fidelity Audio API server. We use this data to ensure that we only send to the server the minimum set of data necessary - i.e. the difference between the data contained on the server about the user and the new data that the client has locally. We use this function here to update the clientside copy of the data that we last transmitted.

    Parameters

    • dataJustTransmitted: HiFiAudioAPIData

      The data that we just transmitted to the High Fidelity Audio API server.

    Returns void

Private _updateUserData

  • _updateUserData(__namedParameters?: { hiFiGain?: number; orientationEuler?: OrientationEuler3D; orientationQuat?: OrientationQuat3D; position?: Point3D; userAttenuation?: number; userRolloff?: number; volumeThreshold?: number }): void
  • Updates the internal copy of the User Data associated with the user associated with this client. Does NOT update the user data on the High Fidelity Audio API server. There are no good reasons for a client to call this function and not update the server User Data, and thus this function is private.

    You can update user orientation by passing Quaternion or Euler orientation representations to this function The quaternion representation is preferred. If both representation are provided, the euler representation is ignored. If only the euler representation is provided, it is then converted immediately to the equivalent quaternion representation. The eulerOrder used for the conversion is the provided by the 'ourAxisConfiguration.eulerOrder'. Euler representation is not used internally anymore in the Hifi API.

    Parameters

    • __namedParameters: { hiFiGain?: number; orientationEuler?: OrientationEuler3D; orientationQuat?: OrientationQuat3D; position?: Point3D; userAttenuation?: number; userRolloff?: number; volumeThreshold?: number } = ...
      • Optional hiFiGain?: number

        This value affects how loud User A will sound to User B at a given distance in 3D space. This value also affects the distance at which User A can be heard in 3D space. Higher values for User A means that User A will sound louder to other users around User A, and it also means that User A will be audible from a greater distance. The new hiFiGain of the user.

      • Optional orientationEuler?: OrientationEuler3D

        The new orientationEuler of the user.

      • Optional orientationQuat?: OrientationQuat3D

        The new orientationQuat of the user.

      • Optional position?: Point3D

        The new position of the user.

      • Optional userAttenuation?: number

        This value affects how far a user's voice will travel in 3D space. The new attenuation value for the user.

      • Optional userRolloff?: number

        This value affects the frequency rolloff for a given user. The new rolloff value for the user.

      • Optional volumeThreshold?: number

        The new volumeThreshold of the user.

    Returns void

addUserDataSubscription

  • Adds a new User Data Subscription to the list of clientside Subscriptions. User Data Subscriptions are used to obtain User Data about other Users. For example, if you set up a User Data Subscription for your own User Data, you can use that subscription to ensure that the data on the High Fidelity Audio API Server is the same as the data you are sending to it from the client.

    To check if a user has disconnected, use HiFiCommunicator.onUsersDisconnected.

    Parameters

    Returns void

connectToHiFiAudioAPIServer

  • connectToHiFiAudioAPIServer(hifiAuthJWT: string, signalingHostURL?: string, signalingPort?: number): Promise<any>
  • Connects to the High Fidelity Audio API server and transmits the initial user data to the server.

    Parameters

    • hifiAuthJWT: string

      This JSON Web Token (JWT) is used by callers to associate a user with a specific High Fidelity Spatial Audio API Server. JWTs are an industry-standard method for securely representing claims between two applications.

      Important information about JWTs:

      • Do not expose JWTs to users! Anyone with access to one of your JWTs will be able to connect to your High Fidelity Spatial Audio API Server.
      • In your application's production environment, each client running your app code should connect to the High Fidelity Spatial Audio Server with a unique JWT. In the case of a Web application, your application server code should generate a JWT associated with the user requesting your Web application.

      To generate a JWT for use with the High Fidelity Audio API:

      1. Head to https://jwt.io/ to find the appropriate library for your langauge. a. For NodeJS applications and Web applications compilied from NodeJS code, we recommend jose.
      2. Using the High Fidelity Audio API Developer Console, obtain your App ID, Space ID, and App Secret.
      3. Create your user's JWT using the appropriate library, passing your App ID, Space ID, and App Secret. Here is an example of what that might look like, using NodeJS and jose:
        hiFiSampleJWT = await new SignJWT({
        "app_id": APP_ID,
        "space_id": SPACE_ID
        })
        .setProtectedHeader({ alg: 'HS256', typ: 'JWT' })
        .sign(crypto.createSecretKey(Buffer.from(APP_SECRET, "utf8")));
        
        Please reference our "Get a JWT" guide for additional context.
      4. Pass the created JWT to connectToHiFiAudioAPIServer().

      As of 2021-01-21, we've added code in this function which, in the browser context, searches for a token URL query parameter and, if a JWT isn't supplied as an argument to this function, uses the value of that token URL query parameter as the JWT. We should remove that later, because we almost certainly don't want this to stay in the API code, but it's very convenient for sample apps for right now.

    • Optional signalingHostURL: string

      An URL that will be used to create a valid WebRTC signaling address at High Fidelity. The passed signalingHostURL parameter should not contain the protocol or port - e.g. server.highfidelity.com - and it will be used to construct a signaling address of the form: wss://${signalingHostURL}:${signalingPort}/?token= If the developer does not pass a signalingHostURL parameter, a default URL will be used instead. See: DEFAULT_PROD_HIGH_FIDELITY_ENDPOINT Reading this parameter from the URL (if needed) should be implemented by the developer as part of the application code.

    • Optional signalingPort: number

      The port to use for making WebSocket connections to the High Fidelity servers. If the developer does not pass a signalingPort parameter, the default (443) will be used instead. See: DEFAULT_PROD_HIGH_FIDELITY_PORT

    Returns Promise<any>

    If this operation is successful, the Promise will resolve with { success: true, audionetInitResponse: <The response to audionet.init from the server in Object format>}. If unsuccessful, the Promise will reject with { success: false, error: <an error message> }.

disconnectFromHiFiAudioAPIServer

  • disconnectFromHiFiAudioAPIServer(): Promise<string>
  • Disconnects from the High Fidelity Audio API. After this call, user data will no longer be transmitted to High Fidelity, the audio input stream will not be transmitted to High Fidelity, and the user will no longer be able to hear the audio stream from High Fidelity.

    Returns Promise<string>

getCommunicatorInfo

  • getCommunicatorInfo(): any
  • Returns any

    A bunch of info about this HiFiCommunicator instantiation, including Server Version.

getConnectionState

  • Returns HiFiConnectionStates

    The current state of the connection to High Fidelity, as one of the HiFiConnectionStates. This will return null if the current state is not available (e.g. if the HiFiCommunicator is still in the process of initializing its underlying HiFiMixerSession).

getOutputAudioMediaStream

  • getOutputAudioMediaStream(): MediaStream
  • Returns MediaStream

    The final mixed audio MediaStream coming from the High Fidelity Audio Server.

setInputAudioMediaStream

  • setInputAudioMediaStream(newInputAudioMediaStream: MediaStream, isStereo?: boolean): Promise<boolean>
  • Use this function to set the MediaStream associated with the user. This MediaStream will be sent up to the High Fidelity Audio Servers and mixed with other users' audio streams. The resultant mixed stream will be sent to all connected clients.

    Be mindful of supplying this stream upon initial connection when you anticipate that the user is using Bluetooth audio input and Bluetooth audio output simultaneously. Many Bluetooth audio devices do not support stereo (spatialized) audio output and microphone audio input simultaneously, including the popular combination of an iPhone and AirPods. Your users may have a better experience if they join the Server in "listen-only" mode - i.e. without microphone input - and then are asked for microphone permission later (which will force their Bluetooth output device into a lower-quality, unspatialized mono mode).

    Parameters

    • newInputAudioMediaStream: MediaStream

      The new MediaStream to send to the High Fidelity Audio Server. If this is set to an undefined value, the existing input stream (if one is set) will be cleared.

    • isStereo: boolean = false

      true if the input stream should be treated as stereo, false for mono (default).

    Returns Promise<boolean>

    true if the new MediaStream was successfully set, false otherwise.

setInputAudioMuted

  • setInputAudioMuted(isMuted: boolean): Promise<boolean>
  • Use this function to set whether input audio stream will have the enabled property of each of its MediaStreamTracks set to false (and an unmuted stream -- the default -- will have the enabled property set to true). This will silence the input, but has specific consequences:

    • If you are using the same MediaStream object in other ways, it will be affected by calling this method. So, if you would like to mute/unmute the input audio stream separately for the High Fidelity audio vs. some other use of it, it is recommended to clone the audio stream separately for each use.
    • The effect is immediate and could result in a click or other audio artifact if there is steady sound at the moment the input is muted.

    An alterative is to set the user's {@link volumeThreshold} to 0, which smoothly gates off the user's input.

    Parameters

    • isMuted: boolean

    Returns Promise<boolean>

    true if the stream was successfully muted/unmuted, false if it was not. (The user should assume that if this returns false, no change was made to the mute (track enabled) state of the stream.)

setOtherUserGainForThisConnection

  • setOtherUserGainForThisConnection(visitIdHash: string, gain: number): Promise<SetOtherUserGainsForThisConnectionResponse>
  • Adjusts the gain of another user for this communicator's current connection only. This is a single user version of HiFiCommunicator.setOtherUserGainsForThisConnection. This can be used to provide a more comfortable listening experience for the client. If you need to perform moderation actions which apply server side, use the Administrative REST API.

    To use this command, the communicator must currently be connected to a space. You can connect to a space using connectToHiFiAudioAPIServer.

    Parameters

    • visitIdHash: string
    • gain: number

      The relative gain to apply to the other user. By default, this is 1.0. The gain can be any value greater or equal to 0.0. For example: a gain of 2.0 will double the loudness of the user, while a gain of 0.5 will halve the user's loudness. A gain of 0.0 will effectively mute the user.

    Returns Promise<SetOtherUserGainsForThisConnectionResponse>

    If this operation is successful, the Promise will resolve with {@link SetOtherUserGainForThisConnectionResponse} with success equal to true. If unsuccessful, the Promise will reject with {@link SetOtherUserGainForThisConnectionResponse} with success equal to false and error set to an error message describing what went wrong.

setOtherUserGainsForThisConnection

  • setOtherUserGainsForThisConnection(otherUserGainMap: OtherUserGainMap): Promise<SetOtherUserGainsForThisConnectionResponse>
  • Adjusts the gain of one or more users for this communicator's current connection only. This can be used to provide a more comfortable listening experience for the client. If you need to perform moderation actions on the server side, use the Administrative REST API.

    To use this command, the communicator must currently be connected to a space. You can connect to a space using connectToHiFiAudioAPIServer.

    Parameters

    • otherUserGainMap: OtherUserGainMap

      The map between hashed visit IDs and the desired adjusted gains of users from the perspective of this client, for this connection only.

      Use addUserDataSubscription and HiFiCommunicator.onUsersDisconnected to keep track of the hashed visit IDs of currently connected users.

      When you subscribe to user data, you will get a list of ReceivedHiFiAudioAPIData objects, which each contain, at minimum, {@link ReceivedHifiAudioAPIData.hashedVisitID}s and {@link ReceivedHifiAudioAPIData.providedUserID}s for each user in the space. By inspecting each of these objects, you can associate a user with their hashed visit ID, if you know their provided user ID.

      The relative gain will be applied to the other user with the matching hashed visit ID. By default, this is 1.0. The gain can be any value greater or equal to 0.0. For example: a gain of 2.0 will double the loudness of the user, while a gain of 0.5 will halve the user's loudness. A gain of 0.0 will effectively mute the user.

    Returns Promise<SetOtherUserGainsForThisConnectionResponse>

    If this operation is successful, the Promise will resolve with {@link SetOtherUserGainsForThisConnectionResponse} with success equal to true. If unsuccessful, the Promise will reject with {@link SetOtherUserGainsForThisConnectionResponse} with success equal to false and error set to an error message describing what went wrong.

startCollectingWebRTCStats

  • startCollectingWebRTCStats(callback: Function): void
  • Start collecting data about the WebRTC connection between Client and Server. Note that the data inside the reports pertains only to payload data internal to the WebRTC connection and does not include total data sent over the wire or received over the wire in your application.

    Parameters

    • callback: Function

      Callback functions will be provided two Array arguments: stats and prevStats. Each of those Array items contains one or more Objects, which are reports of WebRTC stats data, including data such as "a timestamp", "the number of bytes received since the last report" and "current jitter buffer delay".

    Returns void

stopCollectingWebRTCStats

  • stopCollectingWebRTCStats(): void
  • Stop collecting data about the WebRTC connection between Client and Server.

    Returns void

updateUserDataAndTransmit

  • updateUserDataAndTransmit(newUserData: any): string
  • A simple function that calls _updateUserData, followed by _transmitHiFiAudioAPIDataToServer. Developers can call this function as often as they want. This function will update the internal data store of the user's position, orientation, etc. No matter how often developers call this function, the internal data store transmission is rate-limited and will only be sent to the server once every transmitRateLimitTimeoutMS milliseconds. When the internal data store is transmitted, the most up-to-date data will be transmitted.

    Parameters

    • newUserData: any

      The new user data that we want to send to the High Fidelity Audio API server.

    Returns string

    Returns the return value of _transmitHiFiAudioAPIDataToServer.