In Manifest V2, Chrome Extensions have the ability to run the Voice JS SDK in the background when making calls. But with the introduction of Manifest V3, running the Voice JS SDK in the background can only be achieved through service workers. Service workers don't have access to certain features such as DOM, getUserMedia, and audio playback, making it impossible to make calls with previous versions of the SDK.
With this new release, the SDK can now run in a service worker context to listen for incoming calls or initiate outgoing calls. When the call object is created, it can be forwarded to an offscreen document where the SDK has access to all the necessary APIs to fully establish the call. Check our example to see how this works.
Prior versions of the SDK support simultaneous outgoing and incoming calls using different identities. If an incoming call comes in and the Device with the same identity is busy, the active call needs to be disconnected before accepting the incoming call. With this new release of the SDK, multiple incoming calls for the same identity can now be accepted, muted, or put on hold, without disconnecting any existing active calls. This can be achieved by forwarding the incoming call to a different Device instance. See the following new APIs and example for more details.
// Create a Device instance that handles receiving of all incoming calls for the same identity.
const receiverDevice = new Device(token, options);
await receiverDevice.register();
receiverDevice.on('incoming', (call) => {
// Forward this call to a new Device instance using the call.connectToken string.
forwardCall(call.connectToken);
});
// The forwardCall function may look something like the following.
async function forwardCall(connectToken) {
// For every incoming call, we create a new Device instance which we can
// interact with, without affecting other calls.
// IMPORTANT: The token for this new device needs to have the same identity
// as the token used in the receiverDevice.
const device = new Device(token, options);
const call = await device.connect({ connectToken });
// Destroy the device after the call is completed
call.on('disconnect', () => device.destroy());
}- Fixed an issue where an error is thrown when
rtcConstraintsparameter is provided. - Fixed an issue (#118, #210) where certain calls are not ended right away after a page refresh.
- Fixed an issue where
device.register()does not return a promise rejection when the WebSocket fails to connect. Thank you @kamalbennani for your contribution. - Fixed an issue where audio processor insights events are not generated if there is an existing processed stream at the start of a call.
- Added tags to client logs for easier filtering
- Added log statements to API calls and events for debugging purposes
- Fixed an issue where updating token after signaling connection has gone offline causes an Invalid State error.
- Fixed an issue where
Device.Options.logLevelis only accepting anumbertype. With this release,stringsare now also allowed. See Device.Options.logLevel for a list of possible values. - Fixed an issue where
call.mute()does not have an effect while thecall.status()is eitherringingorconnecting. Thank you @zyzmoz for your contribution.
The SDK now includes Audio Processor APIs, enabling access to raw audio input and the ability to modify audio data before sending it to Twilio. With this new feature, the following use cases can now be easily achieved on the client side:
- Background noise removal using a noise cancellation library of your choice
- Music playback when putting the call on hold
- Audio filters
- AI audio classification
- ... and more!
Please visit this page for more details about the Audio Processor APIs.
-
Added a new feature flag
enableImprovedSignalingErrorPrecisionto enhance the precision of errors emitted byDeviceandCallobjects.const token = ...; const device = new Device(token, { enableImprovedSignalingErrorPrecision: true, });
The default value of this option is
false.When this flag is enabled, some errors that would have been described with a generic error code are now described with a more precise error code. With this feature, the following errors now have their own error codes. Please see this page for more details about each error.
-
Device Error Changes
const device = new Device(token, { enableImprovedSignalingErrorPrecision: true, }); device.on('error', (deviceError) => { // the following table describes how deviceError will change with this feature flag });
Device Error Name Device Error Code with Feature Flag Enabled Device Error Code with Feature Flag Disabled GeneralErrors.ApplicationNotFoundError3100153000GeneralErrors.ConnectionDeclinedError3100253000GeneralErrors.ConnectionTimeoutError3100353000MalformedRequestErrors.MissingParameterArrayError3110153000MalformedRequestErrors.AuthorizationTokenMissingError3110253000MalformedRequestErrors.MaxParameterLengthExceededError3110353000MalformedRequestErrors.InvalidBridgeTokenError3110453000MalformedRequestErrors.InvalidClientNameError3110553000MalformedRequestErrors.ReconnectParameterInvalidError3110753000SignatureValidationErrors.AccessTokenSignatureValidationFailed3120253000AuthorizationErrors.NoValidAccountError3120353000AuthorizationErrors.JWTTokenExpirationTooLongError3120753000ClientErrors.NotFound3140453000ClientErrors.TemporarilyUnavilable3148053000ClientErrors.BusyHere3148653000SIPServerErrors.Decline3160353000 -
Call Error Changes
const device = new Device(token, { enableImprovedSignalingErrorPrecision: true, }); const call = device.connect(...); call.on('error', (callError) => { // the following table describes how callError will change with this feature flag });
Call Error Name Call Error Code with Feature Flag Enabled Call Error Code with Feature Flag Disabled GeneralErrors.ConnectionDeclinedError3100231005AuthorizationErrors.InvalidJWTTokenError3120431005AuthorizationErrors.JWTTokenExpiredError3120531005
IMPORTANT: If your application logic currently relies on listening to the generic error code
53000or31005, and you opt into enabling the feature flag, then your applicaton logic needs to be updated to anticipate the new error code when any of the above errors happen. -
- Fixed an issue where, sometimes a TypeError is raised while handling an incoming call under the following circumstances:
- Network interruptions
- updating the token before accepting the call
Updated November 1, 2023
We have identified an issue on Chromium-based browsers running on MacOS 14 (Sonoma) where the audio deteriorates during a call. This issue happens due to the excessive calls to MediaDevices: enumerateDevices() API. With this release, the SDK calls this API only when necessary to avoid audio deterioration.
- Fixed an issue where audio in the Chrome browser is choppy when another application is also using the audio devices.
- Added missing documentation for the following events:
call.on('ringing', handler)call.on('warning', handler)call.on('warning-cleared', handler)device.on('destroyed', handler)
- Fixed an issue where
call.sendMessage()API throws an error if the SDK is imported as an ECMAScript Module (ESM) using the@twilio/voice-sdk/esmpath.
Currently, the SDK is imported as a CommonJS Module (CJS) using the root path @twilio/voice-sdk. With this release, the SDK contains an experimental feature that allows it to be imported as an ECMAScript Module (ESM) using the @twilio/voice-sdk/esm path. As this is an experimental feature, some frameworks using bundlers like Vite and Rollup may not work. Full support for ESM will be available in a future release and will become the default import behavior of the SDK.
Example:
import { Device } from '@twilio/voice-sdk/esm';- Fixed some security vulnerabilities shown by
npm audit. - Removed unused dependencies.
- Replaced deprecated dependencies.
-
Fixed an issue where custom DTMF sounds would not play. With this release, custom DTMF sounds should now play when configured during device initialization.
const device = new Device(token, { sounds: { dtmf8: 'http://mysite.com/8_button.mp3', // Other custom sounds }, // Other options });
-
Fixed an issue where calling
device.updateOptionswould reset thedevice.audio._enabledSoundsstate.
- The SDK now builds on NodeJS versions 16 and above without the
--legacy-peer-depsflag. - Removed usage of NodeJS modules from the SDK and some dependencies. With this change, the SDK should now work with some of the latest frameworks that use the latest versions of bundlers such as Vite and Webpack.
- The AudioPlayer dependency has been incorporated into the SDK as part of a migration. This change fixes an issue where source maps are not properly loaded.
- Removed unnecessary files from the generated npm package.
- Links to source maps are now included in the generated npm package.
- The
wspackage has been moved todevDependencies. - The SDK no longer depends on the
xmlhttprequestnpm package.
Updated: This is now GA as of December 14, 2023
The SDK now allows you to override WebRTC APIs using the following options and events. If your environment supports WebRTC redirection, such as Citrix HDX's WebRTC redirection technologies, your application can use this new beta feature for improved audio quality in those environments.
- Device.Options.enumerateDevices
- Device.Options.getUserMedia
- Device.Options.RTCPeerConnection
- call.on('audio', handler(remoteAudio))
-
Updated the description of Device.updateToken API. It is recommended to call this API after Device.tokenWillExpireEvent is emitted, and before or after a call to prevent a potential ~1s audio loss during the update process.
-
Updated stats reporting to stop using deprecated
RTCIceCandidateStats-ipanddeleted.
-
Fixed an issue where a
TypeErroris thrown after rejecting a call then invokingupdateToken. -
Fixed an issue (#87, #145) where the
PeerConnectionobject is not properly disposed. -
Fixed an issue where
device.audio.disconnect,device.audio.incominganddevice.audio.outgoingdo not have the correct type definitions. -
Fixed an issue where the internal
deviceinfochangeevent is being emitted indefinitely, causing high cpu usage.
- Fixed an issue where a Twilio error is not returned when microphone access is blocked. Thank you @SiimMardus and @ostap0207 for your contribution.
- Fixed an issue where incoming sound will not stop playing after the call is disconnected. Thank you @kamalbennani for your contribution.
This release includes updated DNS names for Twilio Edge Locations. The Voice JS SDK uses these Edge Locations to connect to Twilio’s infrastructure via the parameter Device.Options.edge. The current usage of this parameter does not change as the SDK automatically maps the edge value to the new DNS names.
Additionally, you need to update your Content Security Policies (CSP) if you have it enabled for your application. You also need to update your network configuration such as firewalls, if necessary, to allow connections to the new DNS names and IP addresses.
The SDK can now send and receive custom messages to and from Twilio's backend via the following new Call APIs.
Please visit this page for more details about this feature. Additionally, please see the following for more information on how to send and receive messages on the server.
NOTE: This feature should not be used with PII.
Example
const device = new Device(token, options);
const setupCallHandlers = call => {
call.on('messageReceived', message => messageReceivedHandler(message));
call.on('messageSent', message => messageSentHandler(message));
};
// For outgoing calls
const call = await device.connect();
setupCallHandlers(call);
// For incoming calls
device.on('incoming', call => setupCallHandlers(call));
await device.register();
// For sending a message
const eventSid = call.sendMessage({
content: { foo: 'foo' },
messageType: Call.MessageType.UserDefinedMessage,
});- Fixed an issue where insights data stops getting published after calling
device.updateOptions.
- Ignoring a call will now properly stop the ringing sound
- NPM versioning has been fixed to specify >=12 rather than exactly 12
- Use DOMException instead of DOMError, which has been deprecated
- Removed npm util from the package, instead favoring native functions
The SDK now fully supports Call reconnection. Previously, the SDK only truly supported media reconnection -- if the media connection was lost but the signaling websocket recovered (generally within 10-15 seconds), reconnecting the Call was possible. However, if the websocket was lost, the Call was lost. Now, the SDK is able to reconnect a Call even if the websocket is lost. This means that a Call can potentially be recovered up to 30 seconds or in a network handover event, in which a user switches networks during a call.
When a call has encountered a network error and signaling reconnection has succeeded, the Call object will emit the
reconnected event.
const call = await device.connect(...);
call.on('reconnected', () => { ... });There exists a limitation such that Signaling Reconnection and Edge Fallback are mutually exclusive. To opt-in to the Signaling Reconnection feature, a new option can be passed to the SDK: maxCallSignalingTimeoutMs. If this value is not present in the options object passed to the Device constructor, the default value will be 0.
Using a value of 30000 as an example: while a Call exists, the Device will attempt to reconnect to the edge that the Call was established on for approximately 30 seconds. After the next failure to connect, the Device will use edge-fallback.
const token = ...;
const options = {
...,
edge: ['ashburn', 'sydney'],
maxCallSignalingTimeoutMs: 30000,
};
const device = new Device(token, options);
const call = device.connect(...);
// As an example, the device has connected to the `ashburn` edge.
call.on('accept', () => {
// Starting here, the device will only attempt to connect to `ashburn` if a
// network loss occurs.
// If it cannot connect within `maxCallSignalingTimeoutMs` (in this case 30
// seconds), then it will resort to Edge Fallback.
// The first Edge Fallback attempt will be the next `edge`, in this case
// `sydney`, as specified by the `edge` option passed to the `Device`.
});In order to ensure automatic reconnection is possible at any time, we've also added Device.Event.TokenWillExpire,
which should prompt the application to obtain a new token and call Device.updateToken().
By default, this new event will fire 10 seconds prior to the AccessToken's expiration, prompting the application
to provide a new token. This can be changed by setting Device.Options.tokenRefreshMs to something other than the
default of 10000 ms.
const device = new Device(token, {
tokenRefreshMs: 30000,
});
device.on('tokenWillExpire', () => {
return getTokenViaAjax().then(token => dev.updateToken(token));
});The Twilio Voice JS SDK now supports Twilio Regional. To use a home region, please specify the desired home region in the access token before passing the token to the Twilio Device. This home region parameter should be matched with the appropriate edge parameter when instantiating a Twilio Device. The home region determines the location of your Insights data, as opposed to the edge that your call connects to Twilio through.
If you are using the twilio-node helper library to mint access tokens within your backend, you can specify the au1 home region like so:
const accessToken = new twilio.jwt.AccessToken(
credentials.accountSid,
credentials.apiKeySid,
credentials.apiKeySecret, {
identity,
ttl,
region: 'au1',
},
);
const grant = new VoiceGrant({
outgoingApplicationSid: credentials.twimlAppSid,
incomingAllow: true,
});
accessToken.addGrant(grant);
const device = new Device(accessToken, {
edge: 'sydney',
});Note that the API Key and Secret and TwiML App above must be created within the au1 region.
The current home region can be retrieved from the read-only string Device.home, which contains the currently
connected home region after a successful registration.
After successfully registering, the Device will now have a read-only string, Device.identity, which exposes
the identity passed via token.
- Updated
wsversion to fix a potential security vulnerability. - We now properly clean up all event listeners after
Device.destroy() - We now log a warning rather than an throwing an uncaught promise rejection when Insights fails to post an event.
This patch increment was necessary because the 2.0.0 pilot artifact was erroneously published to npm. It is now removed from npm so that it is not mistakenly used. The first npm artifact will be 2.0.1.
This product, Twilio's JavaScript Voice SDK, is the next version of Twilio's Javascript Client SDK. It is now in GA and we recommend all customers migrate in order to continue receiving future feature additions. For help on migrating from 1.x, see our migration guide.
Note:: These changes are cumulative with the 2.0.0-preview.1 changes below. If you are looking to upgrade from twilio-client.js 1.x, see the 2.0.0-preview.1 section below for the full 2.0 changelog.
An error surrounding re-registration was fixed that occurred when updating the
edge option.
Device.activeCall is no longer available. Instead, the application should keep
a reference to any call that is made using connect or accepted.
const token = '...';
const options = { ... };
const device = new Device(token, options);
await device.register();
const call = await device.connect();Device must now be instantiated before it can be used. Calling Device.setup() will no longer
work; instead, a new Device must be instantiated via new Device(token, options?).
As Connection is an overloaded and ambiguous term, the class has been renamed Call to better indicate what the object represents and be more consistent with Mobile SDKs and our REST APIs.
Device.setup() has been removed, and new Device(...) will not automatically begin
connecting to signaling. There is no need to listen for Device.on('ready'). Instead,
the signaling connection will automatically be acquired in one of two scenarios:
- The application calls
Device.connect(), creating an outbound Call. In this case, the state of the signaling connection will be represented in the Call. - The application calls
Device.register(), which will register the SDK to listen for incoming calls at the identity specified in the AccessToken.
As long as outgoing calls are expected to be made, or incoming calls are expected to be received,
the token supplied to Device should be fresh and not expired. This can be done by setting a
timer in the application to call updateToken with the new token shortly before the prior
token expires. This is important, because signaling connection is lazy loaded and will fail if
the token is not valid at the time of creation.
Example:
const TTL = 600000; // Assuming our endpoint issues tokens for 600 seconds (10 minutes)
const REFRESH_TIMER = TTL - 30000; // We update our token 30 seconds before expiration;
const interval = setInterval(async () => {
const newToken = await getNewTokenViaAjax();
device.updateToken(newToken);
}, REFRESH_TIMER);The Device states have changed. The states were: [Ready, Busy, Offline]. These
have been changed to more accurately and clearly represent the states of the
Device. There are two changes to Device state:
- The states themselves have changed to
[Registered, Registering, Unregistered, Destroyed]. This removes the idea of "Busy" from the state, as technically the Device can have an active Call whether it is registered or not, depending on the use case. The Device will always starty asUnregistered. In this state, it can still make outbound Calls. OnceDevice.register()has been called, this state will change toRegisteringand finallyRegistered. IfDevice.unregister()is called the state will revert toUnregistered. If the signaling connection is lost, the state will transition toRegisteringor `Unregistered' depending on whether or not the connection can be re-established.
The destroyed state represents a Device that has been "destroyed" by calling
Device.destroy. The device should be considered unusable at this point and a
new one should be constructed for further use.
- The busy state has been moved to a boolean,
Device.isBusy. This is a very basic shortcut for the logicreturn !!device.activeConnection.
The events emitted by the Device are represented by the Device.EventName
enum and represent the new Device states:
export enum EventName {
Destroyed = 'destroyed',
Error = 'error',
Incoming = 'incoming',
Unregistered = 'unregistered',
Registering = 'registering',
Registered = 'registered',
}Note that unregistered, registering, and registered have replaced
offline and ready. Although frequently used to represent connected or disconnected,
ready and offline actually were meant to represent registered and unregistered,
which was quite ambiguous and a primary reason for the change.
When the device is destroyed using Device.destroy, a "destroyed" event will
be emitted.
The construction signature and usage of Device has changed. These are the new API signatures:
/**
* Create a new Device. This is synchronous and will not open a signaling socket immediately.
*/
new Device(token: string, options?: Device.Options): Device;
/**
* Promise resolves when the Device has successfully registered.
* Replaces Device.registerPresence()
* Can reject if the Device is unusable, i.e. "destroyed".
*/
async Device.register(): Promise<void>;
/**
* Promise resolves when the Device has successfully unregistered.
* Replaces Device.unregisterPresence()
* Can reject if the Device is unusable, i.e. "destroyed".
*/
async Device.unregister(): Promise<void>;
/**
* Promise resolves when signaling is established and a Call has been created.
* Can reject if the Device is unusable, i.e. "destroyed".
*/
async Device.connect(options?: Device.ConnectOptions): Promise<Call>;const device = new Device(token, { edge: 'ashburn' });
device.on(Device.EventName.Incoming, call => { /* use `call` here */ });
await device.register();const device = new Device(token, { edge: 'ashburn' });
const call = await device.connect({ To: 'alice' });The arguments for Device.connect() and Call.accept() have been standardized
to the following options objects:
interface Call.AcceptOptions {
/**
* An RTCConfiguration to pass to the RTCPeerConnection constructor.
*/
rtcConfiguration?: RTCConfiguration;
/**
* MediaStreamConstraints to pass to getUserMedia when making or accepting a Call.
*/
rtcConstraints?: MediaStreamConstraints;
}interface Device.ConnectOptions extends Call.AcceptOptions {
/**
* A flat object containing key:value pairs to be sent to the TwiML app.
*/
params?: Record<string, string>;
}Note that these now take a MediaStreamConstraints rather than just the audio constraints. For example:
device.connect({ To: 'client:alice' }, { deviceId: 'default' });might be re-written as:
device.connect({
params: { To: 'client:alice' },
rtcConstraints: { audio: { deviceId: 'default' } },
});For backward compatibility, the new error format was attached to the old format under error.twilioError:
class oldError extends Error {
//...
code: number;
message: string;
twilioError: TwilioError;
}The new Error format is:
class TwilioError extends Error {
/**
* A list of possible causes for the Error.
*/
causes: string[];
/**
* The numerical code associated with this Error.
*/
code: number;
/**
* A description of what the Error means.
*/
description: string;
/**
* An explanation of when the Error may be observed.
*/
explanation: string;
/**
* Any further information discovered and passed along at run-time.
*/
message: string;
/**
* The name of this Error.
*/
name: string;
/**
* The original Error received from the external system, if any.
*/
originalError?: Error;
/**
* A list of potential solutions for the Error.
*/
solutions: string[];
}With the transition, the following error codes have changed:
- 31003 -> 53405 | When ICE connection fails
- 31201 -> 31402 | When getting user media fails
- 31208 -> 31401 | When user denies access to user media
- 31901 -> 53000 | When websocket times out in preflight
Previously, Device.setup() could only be used the set options once. Now, we've added
Device.updateOptions(options: Device.Options) which will allow changing
the Device options without instantiating a new Device. Note that the edge cannot be changed
during an active Call.
Example usage:
const options = { edge: 'ashburn' };
const device = new Device(token, options);
// Later...
device.updateOptions({ allowIncomingWhileBusy: true });The resulting (non-default) options would now be:
{
allowIncomingWhileBusy: true,
edge: 'ashburn',
}This function will throw with an InvalidStateError if the Device has been
destroyed beforehand.
The SDK now uses the loglevel module. This exposes
several new features for the SDK, including the ability to intercept log messages with custom
handlers and the ability to set logging levels after instantiating a Device. To get an instance
of the loglevel Logger class used internally by the SDK:
import { Logger } from '@twilio/voice-sdk';
...
Logger.setLogLevel('DEBUG');Please see the original loglevel project for more
documentation on usage.
- Removed
Connection.mediaStream. To access the MediaStreams, useConnection.getRemoteStream()andConnection.getLocalStream() - Removed
Connection.messagein favor of the newerConnection.customParameters. Where.messagewas an Object,.customParametersis aMap. - Removed the following private members from the public interface:
Connection.optionsConnection.pstreamConnection.sendHangup
- Fixed
Connection.on('cancel')logic so that we no longer emitcancelin response toConnection.ignore().
Some deprecated Device options have been removed. This includes:
enableIceRestartenableRingingStatefakeLocalDtmf
The above three removed options are now assumed true. The new Device.Options interface is now:
export interface Options {
allowIncomingWhileBusy?: boolean;
appName?: string;
appVersion?: string;
audioConstraints?: MediaTrackConstraints | boolean;
closeProtection?: boolean | string;
codecPreferences?: Connection.Codec[];
disableAudioContextSounds?: boolean;
dscp?: boolean;
edge?: string[] | string;
forceAggressiveIceNomination?: boolean;
maxAverageBitrate?: number;
rtcConfiguration?: RTCConfiguration;
sounds?: Partial<Record<Device.SoundName, string>>;
}The formula used to calculate the mean-opinion score (MOS) has been fixed for extreme network conditions. These fixes will not affect scores for nominal network conditions.