Options
All
  • Public
  • Public/Protected
  • All
Menu

Class HiFiCommunicator

This class exposes properties and methods useful for communicating from the High Fidelity Audio API Client to the High Fidelity Audio API Server.

Hierarchy

  • HiFiCommunicator

Index

Constructors

constructor

  • Constructor for the HiFiCommunicator object. Once you have created a HiFiCommunicator, you can use the setInputAudioMediaStream method to assign an input audio stream to the connection, and once the connection has been established, use the getOutputAudioMediaStream method to retrieve the output audio from the server.

    Parameters

    • __namedParameters: { connectionRetryAndTimeoutConfig?: ConnectionRetryAndTimeoutConfig; customSTUNandTURNConfig?: CustomSTUNandTURNConfig; initialHiFiAudioAPIData?: HiFiAudioAPIData; onConnectionStateChanged?: Function; onMuteChanged?: OnMuteChangedCallback; onUsersDisconnected?: Function; transmitRateLimitTimeoutMS?: number; userDataStreamingScope?: HiFiUserDataStreamingScopes; webrtcSessionParams?: WebRTCSessionParams; worldFrameConfig?: WorldFrameConfiguration } = ...
      • Optional connectionRetryAndTimeoutConfig?: ConnectionRetryAndTimeoutConfig

        Settings for configuring auto-reconnect behavior and the amount of time spent trying to connect before giving up. See ConnectionRetryAndTimeoutConfig for the format of this object. Values that are omitted from the passed object will be set to their defaults.

      • Optional customSTUNandTURNConfig?: CustomSTUNandTURNConfig

        Cannot be set later. This object can be used if specific STUN and TURN server information needs to be provided for negotiating the underlying WebRTC connection. By default, High Fidelity's TURN server will be used, which should suffice for most operations. This is primarily useful for testing or for using a commercial TURN server provider for dealing with particularly challenging client networks/firewalls. See CustomSTUNandTURNConfig for the format of this object (note that all values must be provided when setting this).

      • Optional initialHiFiAudioAPIData?: HiFiAudioAPIData

        The initial position, orientation, etc of the user.

      • Optional onConnectionStateChanged?: Function

        A function that will be called when the connection state to the High Fidelity Audio API Server changes. See HiFiConnectionStates.

      • Optional onMuteChanged?: OnMuteChangedCallback

        A function that will be called when the mute state of the client has changed, for example when muted by an admin. See OnMuteChangedCallback for the information this function will receive.

      • Optional onUsersDisconnected?: Function

        A function that will be called when a peer disconnects from the Space.

      • Optional transmitRateLimitTimeoutMS?: number

        User Data updates will not be sent to the server any more frequently than this number in milliseconds.

      • Optional userDataStreamingScope?: HiFiUserDataStreamingScopes

        Cannot be set later. See HiFiUserDataStreamingScopes.

      • Optional webrtcSessionParams?: WebRTCSessionParams

        Cannot be set later. Extra parameters used for configuring the underlying WebRTC connection to the API servers. These settings are not frequently used; they are primarily for specific jitter buffer configurations.

      • Optional worldFrameConfig?: WorldFrameConfiguration

        The WorldFrameConfiguration cannot be set later. Used to transform between World and HiFi coordinate frames. See WorldFrameConfiguration for details.

    Returns HiFiCommunicator

Properties

Private _connectionRetryAndTimeoutConfig

_connectionRetryAndTimeoutConfig: ConnectionRetryAndTimeoutConfig

Private Optional _coordFrameUtil

_coordFrameUtil: HiFiCoordinateFrameUtil

If the World coordinate system is NOT compatible with the HiFi coordindate frame used by the mixer then configure a HiFiCoordinateFrameUtil to transform to and from HiFi-frame. All position/orientation data will automatically transformed to/from the HiFi coordinate frame by the HiFiMixerSession for on-the-wire data.

The World-frame is compatible iff: (1) It is right-handed (2) It uses the Y-axis (positive or negative, doesn't matter) for the UP direction.

For all other cases create a WorldFrameConfiguration and pass it to the HiFiCommunicator constructor.

Private _currentHiFiAudioAPIData

_currentHiFiAudioAPIData: HiFiAudioAPIData

Private _currentHiFiConnectionState

_currentHiFiConnectionState: HiFiConnectionStates = ...

Stores the current HiFi Connection State, which is an abstraction separate from the individual states of the WebRTC (RAVI Session) state and the RAVI Signaling State. The connection state starts at "Disconnected" until a connection attempt has been made.

Private Optional _customSTUNandTURNConfig

_customSTUNandTURNConfig: CustomSTUNandTURNConfig

Private _failureNotificationPending

_failureNotificationPending: HiFiConnectionAttemptResult

Private _inputAudioMediaStream

_inputAudioMediaStream: MediaStream

Private _lastTransmittedHiFiAudioAPIData

_lastTransmittedHiFiAudioAPIData: HiFiAudioAPIData

Private _mixerSession

_mixerSession: HiFiMixerSession

Private _rejectOpen

_rejectOpen: Function

Private _resolveOpen

_resolveOpen: Function

Private _retryTimerInProgress

_retryTimerInProgress: any

Private _timers

_timers: any = ...

Private _userDataSubscriptions

_userDataSubscriptions: UserDataSubscription[]

Private Optional _webRTCSessionParams

_webRTCSessionParams: WebRTCSessionParams

onConnectionStateChanged

onConnectionStateChanged: Function

This is a function that will get called when the "connection state" changes. It should be set when the HiFiCommunicator object is first constructed. (Note that if a connection state re-triggers -- e.g. if a "Closed" connection is closed again -- this will not be called. It only gets called when the new state is different than the previous state.)

onUsersDisconnected

onUsersDisconnected: Function

transmitRateLimitTimeoutMS

transmitRateLimitTimeoutMS: number

Methods

Private _cancelRetriedConnectionAttempts

  • _cancelRetriedConnectionAttempts(): void
  • This method will cancel any connection attempts that are in progress, and move the HiFiCommunicator state to "Failed". It is meant to be called when an attempt to reconnect times out.

    Returns void

Private _cleanUpHiFiAudioAPIDataHistory

  • _cleanUpHiFiAudioAPIDataHistory(): void
  • Normally, we try to limit the amount of data we transmit to the High Fidelity Audio API server, by remembering what we sent. See _updateUserData for more information on how this is done.

    This function exists to handle any scenarios of remembering too much sent data. It is called just after data is succesfully sent, when data is known to no longer be needed.

    Returns void

Private _connectToHiFiMixer

  • _connectToHiFiMixer(): void
  • Attempt to make a connection based on all of the information that we've gathered so far. This should only be called by or after connectToHiFiAudioAPIServer(), because it relies on connectToHiFiAudioAPIServer() having set up this._mixerSession appropriately.

    Returns void

Private _handleUserDataUpdates

  • Ingests user data updates from the server and, if relevant, calls the relevant callback functions associated with the User Data Subscriptions. See addUserDataSubscription.

    Parameters

    • newUserDataFromServer: ReceivedHiFiAudioAPIData[]

      Contains all of the new user data most recently received from the server.

    Returns void

Private _manageConnection

  • This method manages the connection to the High Fidelity servers.

    Calling this method with the "Connecting" or "Reconnecting" states will trigger it to call the _connectToHiFiMixer() method. This method handles when to resolve or reject the Promise that got opened by connectToHiFiAudioAPIServer()

    This method also manages things like auto-retries, when to call (and not to call) the customer-supplied onConnectionStateChanged function, and keeping track of the current "overarching meta-state" of the communicator as a whole (which might be different from the current state of the MixerSession, due to retries and failures and the like). This method is passed directly into the MixerSession to serve as its state change handler, but is also used internally to, well, manage the connection.

    Parameters

    • newState: HiFiConnectionStates

      The desired new state. If this is "Connecting" or "Reconnecting", the appropriate method will be called to kick off a connection attempt. If this is "Failed" or "Disconnected", retries will be attempted if they're configured.

    • Optional message: HiFiConnectionAttemptResult

      An optional message to include as part of the state change. This will be communicated to users as part of the Reject or Resolve of the connection opening Promise.

    Returns void

Private _maybeClearRateLimitTimeout

  • _maybeClearRateLimitTimeout(): void
  • Clears the clientside rate limit timeout used to prevent user data from being sent to the High Fidelity Audio API server too often.

    Returns void

Private _onUsersDisconnected

  • A simple wrapper function called by our instantiation of HiFiMixerSession that calls the user-provided onUsersDisconnected() function if one exists. Library users can provide an onUsersDisconnected() callback function when instantiating the HiFiCommunicator object, or by setting HiFiCommunicator.onUsersDisconnected after instantiation.

    Parameters

    Returns void

Private _transmitHiFiAudioAPIDataToServer

  • _transmitHiFiAudioAPIDataToServer(forceTransmit?: boolean): any
  • Formats the local user data properly, then sends that user data to the High Fidelity Audio API server. This transfer is rate limited.

    There is no reason a library user would need to call this function without also simultaneously updating User Data, so this function is private.

    Parameters

    • Optional forceTransmit: boolean

      true if we should ignore the clientside rate limiter and send the data regardless of its status; false otherwise.

    Returns any

    If this operation is successful, returns { success: true, rawDataTransmitted: <the raw data that was transmitted to the server>}. If unsuccessful, returns { success: false, error: <an error message> }.

Private _updateLastTransmittedHiFiAudioAPIData

  • _updateLastTransmittedHiFiAudioAPIData(dataJustTransmitted: HiFiAudioAPIData): void
  • We keep a clientside copy of the data that we last transmitted to the High Fidelity Audio API server. We use this data to ensure that we only send to the server the minimum set of data necessary - i.e. the difference between the data contained on the server about the user and the new data that the client has locally. We use this function here to update the clientside copy of the data that we last transmitted.

    Parameters

    • dataJustTransmitted: HiFiAudioAPIData

      The data that we just transmitted to the High Fidelity Audio API server.

    Returns void

Private _updateStateAndCallUserStateChangeHandler

  • This method will handle updating the _currentHiFiConnectionState and notifying the user's callback (when we're ready for that to happen). This will also resolve or reject the Promise made by the connectToHiFiAudioAPIServer() method. All updates to this._currentHiFiConnectionState should go through this method unless there's a really good reason (e.g. _cancelRetriedConnectionAttempts)

    Parameters

    Returns void

Private _updateUserData

  • _updateUserData(__namedParameters?: { hiFiGain?: number; orientation?: Quaternion; position?: Point3D; userAttenuation?: number; userRolloff?: number; volumeThreshold?: number }): void
  • Updates the internal copy of the User Data associated with the user associated with this client. Does NOT update the user data on the High Fidelity Audio API server. There are no good reasons for a client to call this function and not update the server User Data, and thus this function is private.

    Parameters

    • __namedParameters: { hiFiGain?: number; orientation?: Quaternion; position?: Point3D; userAttenuation?: number; userRolloff?: number; volumeThreshold?: number } = ...
      • Optional hiFiGain?: number

        This value affects how loud User A will sound to User B at a given distance in 3D space. This value also affects the distance at which User A can be heard in 3D space. Higher values for User A means that User A will sound louder to other users around User A, and it also means that User A will be audible from a greater distance. The new hiFiGain of the user.

      • Optional orientation?: Quaternion

        The new orientation of the user (in Quaternion form)

      • Optional position?: Point3D

        The new position of the user.

      • Optional userAttenuation?: number

        This value affects how far a user's voice will travel in 3D space. Setting this to NaN will use the space default attenuation, or, if zones are defined for the space, the attenuation settings at the user's position.

        COMPATIBILITY WARNING: Currently, setting userAttenuation to 0 will also reset its value to the space/zone default attenuation. In the future, the value of userAttenuation will only be reset if it is set to NaN. A userAttenuation set to 0 will in the future be treated as a "broadcast mode", making the user audible throughout the entire space. If your spatial audio client application is currently resetting userAttenuation by setting it to 0, please change it to set userAttenuation to NaN instead, in order for it to continue working with future versions of High Fidelity's Spatial Audio API.

      • Optional userRolloff?: number

        This value affects the frequency rolloff for a given user. Setting this to NaN or 0 will use the space default rolloff, or, if zones are defined for the space, the frequency rolloff settings at the user's position.

        COMPATIBILITY WARNING: Currently, setting userRolloff to 0 will also reset its value to the space/zone default rolloff In the future, the value of userRolloff will only be reset if it is set to NaN A userRolloff set to 0 will in the future be treated as a valid frequency rolloff value, which will cause the user's sound to become muffled over a short distance. If your spatial audio client application is currently resetting userRolloff by setting it to 0, please change it to set userRolloff to NaN instead, in order for it to continue working with future versions of High Fidelity's Spatial Audio API.

      • Optional volumeThreshold?: number

        The new volumeThreshold of the user. Setting this to NaN will reset its value to the space default volume threshold.

        COMPATIBILITY WARNING: Currently, setting volumeThreshold to null will also reset its value to the space default volume threshold. In the future, the value of volumeThreshold will only be reset if it is set to NaN. A volumeThreshold set to null will in the future will be treated as if volumeThreshold is not supplied. If your spatial audio client application is currently resetting volumeThreshold by setting it to null, please change it to set volumeThreshold to NaN instead, in order for it to continue working with future versions of High Fidelity's Spatial Audio API.

    Returns void

addUserDataSubscription

  • Adds a new User Data Subscription to the list of clientside Subscriptions. User Data Subscriptions are used to obtain User Data about other Users. For example, if you set up a User Data Subscription for your own User Data, you can use that subscription to ensure that the data on the High Fidelity Audio API Server is the same as the data you are sending to it from the client.

    To check if a user has disconnected, use HiFiCommunicator.onUsersDisconnected.

    Parameters

    Returns void

connectToHiFiAudioAPIServer

  • connectToHiFiAudioAPIServer(hifiAuthJWT: string, signalingHostURL?: string, signalingPort?: number): Promise<HiFiConnectionAttemptResult>
  • Connects to the High Fidelity Audio API server and transmits the initial user data to the server.

    Parameters

    • hifiAuthJWT: string

      This JSON Web Token (JWT) is used by callers to associate a user with a specific High Fidelity Spatial Audio API Server. JWTs are an industry-standard method for securely representing claims between two applications.

      Important information about JWTs:

      • Do not expose JWTs to users! Anyone with access to one of your JWTs will be able to connect to your High Fidelity Spatial Audio API Server.
      • In your application's production environment, each client running your app code should connect to the High Fidelity Spatial Audio Server with a unique JWT. In the case of a Web application, your application server code should generate a JWT associated with the user requesting your Web application.

      To generate a JWT for use with the High Fidelity Audio API:

      1. Head to https://jwt.io/ to find the appropriate library for your langauge. a. For NodeJS applications and Web applications compilied from NodeJS code, we recommend jose.
      2. Using the High Fidelity Audio API Developer Console, obtain your App ID, Space ID, and App Secret.
      3. Create your user's JWT using the appropriate library, passing your App ID, Space ID, and App Secret. Here is an example of what that might look like, using NodeJS and jose:
        hiFiSampleJWT = await new SignJWT({
        "app_id": APP_ID,
        "space_id": SPACE_ID
        })
        .setProtectedHeader({ alg: 'HS256', typ: 'JWT' })
        .sign(crypto.createSecretKey(Buffer.from(APP_SECRET, "utf8")));
        
        Please reference our "Get a JWT" guide for additional context.
      4. Pass the created JWT to connectToHiFiAudioAPIServer().
    • Optional signalingHostURL: string

      An URL that will be used to create a valid WebRTC signaling address at High Fidelity. The passed signalingHostURL parameter should not contain the protocol or port - e.g. server.highfidelity.com - and it will be used to construct a signaling address of the form: wss://${signalingHostURL}:${signalingPort}/?token= If the developer does not pass a signalingHostURL parameter, a default URL will be used instead. See: DEFAULT_PROD_HIGH_FIDELITY_ENDPOINT Reading this parameter from the URL (if needed) should be implemented by the developer as part of the application code.

    • Optional signalingPort: number

      The port to use for making WebSocket connections to the High Fidelity servers. If the developer does not pass a signalingPort parameter, the default (443) will be used instead. See: DEFAULT_PROD_HIGH_FIDELITY_PORT

    Returns Promise<HiFiConnectionAttemptResult>

    If this operation is successful, the Promise will resolve with { success: true, audionetInitResponse: <The response to audionet.init from the server in Object format>}. If unsuccessful, the Promise will reject with { success: false, error: <an error message> }.

disconnectFromHiFiAudioAPIServer

  • disconnectFromHiFiAudioAPIServer(): Promise<string>
  • Disconnects from the High Fidelity Audio API. After this call, user data will no longer be transmitted to High Fidelity, the audio input stream will not be transmitted to High Fidelity, and the user will no longer be able to hear the audio stream from High Fidelity.

    Returns Promise<string>

getCommunicatorInfo

  • getCommunicatorInfo(): any
  • Returns any

    A bunch of info about this HiFiCommunicator instantiation, including Server Version.

getConnectionState

  • Returns HiFiConnectionStates

    The current state of the connection to High Fidelity, as one of the HiFiConnectionStates. This will return undefined if the current state is not available (e.g. if the HiFiCommunicator is still in the process of initializing its underlying HiFiMixerSession).

getOutputAudioMediaStream

  • getOutputAudioMediaStream(): MediaStream
  • Returns MediaStream

    The final mixed audio MediaStream coming from the High Fidelity Audio Server.

setInputAudioMediaStream

  • setInputAudioMediaStream(newInputAudioMediaStream: MediaStream, isStereo?: boolean): Promise<boolean>
  • Use this function to set the MediaStream associated with the user. This MediaStream will be sent up to the High Fidelity Audio Servers and mixed with other users' audio streams. The resultant mixed stream will be sent to all connected clients.

    Be mindful of supplying this stream upon initial connection when you anticipate that the user is using Bluetooth audio input and Bluetooth audio output simultaneously. Many Bluetooth audio devices do not support stereo (spatialized) audio output and microphone audio input simultaneously, including the popular combination of an iPhone and AirPods. Your users may have a better experience if they join the Server in "listen-only" mode - i.e. without microphone input - and then are asked for microphone permission later (which will force their Bluetooth output device into a lower-quality, unspatialized mono mode).

    Parameters

    • newInputAudioMediaStream: MediaStream

      The new MediaStream to send to the High Fidelity Audio Server. If this is set to an undefined value, the existing input stream (if one is set) will be cleared.

    • isStereo: boolean = false

      Most microphones are mono and the default value for isStereo (false) will produce the usual spatial sound. If the input stream is a stereo music stream or a stereo microphone, the application can specify true so that everyone will hear this input’s left and right channels from the respective sides. However, doing so effectively gives this stream a position, but no orientation: the left/right spatialization that everyone will hear is whatever is in the original source, regardless of the orientation of the source.

    Returns Promise<boolean>

    true if the new MediaStream was successfully set, false otherwise.

setInputAudioMuted

  • setInputAudioMuted(isMuted: boolean): Promise<boolean>
  • Use this function to set whether input audio stream will have the enabled property of each of its MediaStreamTracks set to false (and an unmuted stream -- the default -- will have the enabled property set to true). This will silence the input, but has specific consequences:

    • If you are using the same MediaStream object in other ways, it will be affected by calling this method. So, if you would like to mute/unmute the input audio stream separately for the High Fidelity audio vs. some other use of it, it is recommended to clone the audio stream separately for each use.
    • The effect is immediate and could result in a click or other audio artifact if there is steady sound at the moment the input is muted.

    An alterative is to set the user's {@link volumeThreshold} to 0, which smoothly gates off the user's input.

    Parameters

    • isMuted: boolean

    Returns Promise<boolean>

    true if the stream was successfully muted/unmuted, false if it was not. (The user should assume that if this returns false, no change was made to the mute (track enabled) state of the stream.)

setOtherUserGainForThisConnection

  • Adjusts the gain of another user for this communicator's current connection only. This is a single user version of HiFiCommunicator.setOtherUserGainsForThisConnection. This can be used to provide a more comfortable listening experience for the client. If you need to perform moderation actions which apply server side, use the Administrative REST API.

    To use this command, the communicator must currently be connected to a space. You can connect to a space using connectToHiFiAudioAPIServer.

    Parameters

    • visitIdHash: string
    • gain: number

      The relative gain to apply to the other user. By default, this is 1.0. The gain can be any value greater or equal to 0.0. For example: a gain of 2.0 will double the loudness of the user, while a gain of 0.5 will halve the user's loudness. A gain of 0.0 will effectively mute the user.

    Returns Promise<SetOtherUserGainsForThisConnectionResponse>

    If this operation is successful, the Promise will resolve with SetOtherUserGainForThisConnectionResponse with success equal to true. If unsuccessful, the Promise will reject with SetOtherUserGainForThisConnectionResponse with success equal to false and error set to an error message describing what went wrong.

setOtherUserGainsForThisConnection

  • Adjusts the gain of one or more users for this communicator's current connection only. This can be used to provide a more comfortable listening experience for the client. If you need to perform moderation actions on the server side, use the Administrative REST API.

    To use this command, the communicator must currently be connected to a space. You can connect to a space using connectToHiFiAudioAPIServer.

    Parameters

    • otherUserGainMap: OtherUserGainMap

      The map between hashed visit IDs and the desired adjusted gains of users from the perspective of this client, for this connection only.

      Use addUserDataSubscription and HiFiCommunicator.onUsersDisconnected to keep track of the hashed visit IDs of currently connected users.

      When you subscribe to user data, you will get a list of ReceivedHiFiAudioAPIData objects, which each contain, at minimum, {@link ReceivedHifiAudioAPIData.hashedVisitID}s and {@link ReceivedHifiAudioAPIData.providedUserID}s for each user in the space. By inspecting each of these objects, you can associate a user with their hashed visit ID, if you know their provided user ID.

      The relative gain will be applied to the other user with the matching hashed visit ID. By default, this is 1.0. The gain can be any value greater or equal to 0.0. For example: a gain of 2.0 will double the loudness of the user, while a gain of 0.5 will halve the user's loudness. A gain of 0.0 will effectively mute the user.

    Returns Promise<SetOtherUserGainsForThisConnectionResponse>

    If this operation is successful, the Promise will resolve with SetOtherUserGainsForThisConnectionResponse with success equal to true. If unsuccessful, the Promise will reject with SetOtherUserGainsForThisConnectionResponse with success equal to false and error set to an error message describing what went wrong.

startCollectingWebRTCStats

  • startCollectingWebRTCStats(callback: Function): void
  • Start collecting data about the WebRTC connection between Client and Server. Note that the data inside the reports pertains only to payload data internal to the WebRTC connection and does not include total data sent over the wire or received over the wire in your application.

    Parameters

    • callback: Function

      Callback functions will be provided two Array arguments: stats and prevStats. Each of those Array items contains one or more Objects, which are reports of WebRTC stats data, including data such as "a timestamp", "the number of bytes received since the last report" and "current jitter buffer delay".

    Returns void

stopCollectingWebRTCStats

  • stopCollectingWebRTCStats(): void
  • Stop collecting data about the WebRTC connection between Client and Server.

    Returns void

updateUserDataAndTransmit

  • updateUserDataAndTransmit(newUserData: any): string
  • A simple function that calls _updateUserData, followed by _transmitHiFiAudioAPIDataToServer. See HiFiAudioAPIData for what data can be sent to the High Fidelity Audio API Server. Developers can call this function as often as they want. This function will update the internal data store of the user's position, orientation, etc. No matter how often developers call this function, the internal data store transmission is rate-limited and will only be sent to the server once every transmitRateLimitTimeoutMS milliseconds. When the internal data store is transmitted, the most up-to-date data will be transmitted.

    Parameters

    • newUserData: any

      The new user data that we want to send to the High Fidelity Audio API server.

    Returns string

    Returns the return value of _transmitHiFiAudioAPIDataToServer.