Skip to content
This repository has been archived by the owner on May 6, 2020. It is now read-only.

Clustering

Jump to bottom
mpermar edited this page May 6, 2012 · 5 revisions

A Rayo Cluster is made of one or many individual Rayo Nodes which are coordinated by one or more Rayo Gateways. The Rayo Gateways communicate with Rayo applications through an XMPP server. Unlike with individual servers on a Rayo Cluster the XMPP Server never communicates directly with the Rayo Nodes but instead goes through the Rayo Gateway which takes care of most of the non functional requirements like load balancing, security, failover, etc.

The following picture displays the architecture of a Rayo Gateway:

Clustering

The picture above shows a relatively complex scenario with several Rayo Nodes and Rayo Gateways. However, a Rayo Cluster can be much simpler:

Simpler clustering

Rayo Gateway

The Rayo Gateway is the main element that differentiates a Rayo Cluster from a stand-alone Rayo deployment. The Rayo Gateway is an intermediate component that is in charge of coordinating the communication from applications to Rayo Nodes and from Rayo Nodes to applications. The Rayo Gateway has several responsabilities:

  • Load Balancing outgoing calls. The Rayo Gateway is in charge of load balancing outgoing call requests from Rayo applications through the different Rayo Nodes.
  • Load Balancing incoming calls. The Rayo Gateay load balances incoming calls across the different resources of the registered Rayo applications.
  • Message routing. A Rayo Gateway has to route incoming Rayo commands to the appropriate nodes and incoming Rayo Node events to the appropriate client applications.
  • Distributed database. A Rayo Gateway contains a distributed database that stores a complete snapshot of what is going on through the Rayo cluster. This database is used for message routing, monitoring, statistics or resource management.
  • Scalability. At any time the Rayo Gateway can enroll more Rayo Nodes in the Rayo cluster providing a great degree of scalability.
  • Failover. A Rayo Gateway deals also with is in charge of cluster failover management. The Rayo Gateway for example will transparently retry failed operations like dial requests or will automatically take out the Rayo Nodes that keep failing repeatedly from the cluster.
  • Security management. The Rayo Gateway will validate permissions on incoming/outgoing calls as on all the different rayo commands and events for each Rayo application if necessary.

Downloads

The latest version of the Rayo Gateway web application can be downloaded from our continuous integration server. Just download the rayo-gateway-bxxx.war file and drop it into your Prism's apps folder.

Message Routing

The Rayo Gateway communicates both with Rayo Nodes and the XMPP Server through S2S XMPP connections. To differentiate from the messages that come from Rayo Nodes and from client applications, the Rayo Gateway needs to have associated two separate domain names. It is what in Rayo-terms we would call external and internal interfaces:

Incoming call routing

The image above shows how the two Rayo Gateway interfaces would be using for routing a Rayo event from the Rayo Node. The workflow is very simple:

  1. An event is generated in a Rayo Node. In this case an event.
  2. The Rayo Node forwards the event to the Rayo Gateway using the internal interface.
  3. The Rayo Gateway notices that a message comes to its internal interface and therefore it knows that it comes from a Rayo Node. It analyzes the nature of the message and finds that it is an offer.
  4. The Rayo Gateway forwards the event to a Rayo application. To do that, first it needs to change the to and from fields from the original message. In this case the to field is the destination application's jid and the from field is the Rayo Gateway's external interface.
  5. The application receives the event. The application will always interact with the Rayo Cluster using the external interface.

The following example shows how an outbound call would be:

Outgoing call routing

As you might have notice, the above diagram is very similar to the previous one. The main difference is the direction of the messages and again the values of the to and from fields. The workflow is the following:

  1. An application sends a request to the Rayo Cluster. It will use the external interface.
  2. The Rayo Gateway notices that the message comes to its external interface and therefore it knows that it comes from a client application. It analyzes the nature of the message and finds that is a dial request.
  3. The Rayo Gateway finds an available Rayo Node and forwards the message to it. But before forwarding, it changes the from field to its own internal interface and the to field to the match the Rayo Node's domain name.
  4. The Rayo Node receives the message and starts the outbound call.

It will be obvious from the examples above that to get a Rayo cluster working properly it is vital to define correctly the external and internal interfaces. You will find in the section below how to configure your Rayo cluster.

Configuration

Configuring a Rayo Cluster is not much different from configuring an individual Rayo Node. Check out the Single Node and Clustering configuration reference for more information.

Licensing

A Rayo Gateway does not need any SIP functionality and therefore when running on Prism does not need any SIP or media server licenses. However, when running on Prism you will need enough XMPP ports to handle all the traffic required by your Rayo applications.

Gateway's distributed database

As it was mentioned in the introduction, a Rayo Cluster has a distributed storage that keeps a real-time view of what is going on in the cluster. This storage knows for example how many live calls are there in the cluster and in which nodes they are running, how many active client applications are there, how many nodes are available and their occupation, etc.

The default distributed storage system that we ship with our Rayo Gateway implementation is based on Cassandra, the distributed transactional NoSQL database from Apache. By default your Rayo Gateway should be pointing to localhost and 9160 which is Cassandra's default port. If for whatever reason you need to change this values then you only need to edit the cassandra.properties file located at WEB-INF.

Browsing the distributed storage

There is many advantages on using Cassandra. However one of the less known is that there is actually very nice Open Source tools like Virgil that can be used to browse (actually it can be used to do many other things as it provides a REST interface to the Cassandra storage) the actual Cassandra storage.

If you have got your Rayo Cluster from Voxeolabs then chances are you already got Virgil installed. If that's the case then browse to http://your-rayo-gateway:9080 to see if Virgil is there listening. If not, then perhaps is listening at other port or you could ask us to install it. Or you can also try to install it yourself. There is instructions at Virgil's website. Here is an example on how the Rayo Gateway's distributed storage can be seen from Virgil:

Browsing Cassandra

Managing Rayo Applications

To use your shiny and new Rayo cluster you should first register some applications on it and add some addresses to these applications. There is actually two different ways to do this:

  1. Use Tropo's Provisioning API. If you have access to an instance of Tropo's Provisioning API, then you can use the API to automatically populate your Cluster's datastore with applications, addresses, etc.
  2. Directly add applications and addresses with the Gateway's admnistration interface.

With regards to the Provisioning API integration, there is information in this document on how to integrate with Tropo's provisioning API.

If you have no access to a Provisioning API installation, then you can still add the applications and addresses yourself by using the administrative interface. The admin interface is described in the next section, however I will put here the most relevant commands that can be used to create/remove applications and add/remove addresses:

Registering a Rayo application in the cluster

The following example registers an application named voxeolabs on the platform staging with a JID voxeolabs@xmpp-ext.qa.voxeolabs.net:

curl http://localhost:8080/rayo/jmx/exec/com.rayo.gateway:Type=Admin,name=Admin/registerApplication/staging/voxeolabs/voxeolabs@xmpp-ext.qa.voxeolabs.net

Adding an address to an application

The following example adds a SIP address sip:usera@node-ext.qa.voxeolabs.net to the application with JID voxeolabs@xmpp-ext.qa.voxeolabs.net:

curl http://localhost:8080/rayo/jmx/exec/com.rayo.gateway:Type=Admin,name=Admin/registerAddress/voxeolabs@xmpp-ext.qa.voxeolabs.net/sip:usera@node-ext.qa.voxeolabs.net

Removing an address

The following example adds unlinks the SIP address sip:usera@node-ext.qa.voxeolabs.net from whatever application it may be linked to:

curl http://localhost:8080/rayo/jmx/exec/com.rayo.gateway:Type=Admin,name=Admin/unregisterAddress/sip:usera@node-ext.qa.voxeolabs.net

Removing an application

The following example removes the application with a JID voxeolabs@xmpp-ext.qa.voxeolabs.net:

curl http://localhost:8080/rayo/jmx/exec/com.rayo.gateway:Type=Admin,name=Admin/unregisterApplication/voxeolabs@xmpp-ext.qa.voxeolabs.net

Administration

As with Rayo Nodes, the Rayo Gateway can be administered via JMX or via a nice REST/JSON api. We use a wonderful library called Jolokia for doing this. The Rayo Monitoring explains the basic concepts for monitoring Rayo Nodes which are also applicable to monitoring a Rayo Gateway.

MBean summary

Following are some of the most interesting MBeans that can be used to administrate and monitor your Rayo Gateway:

Gateway

The Gateway MBean exposes information about the state of the Rayo Gateway. Most of this information comes from Cassandra itself (so you can query it either via this MBean or via a tool like Virgil described above).

JMX URL: com.rayo.gateway:Type=Gateway

Attributes exposed:

  • Platforms: Returns information about the platforms registered in this Rayo cluster (e.g. production, staging).
  • Nodes: Returns information about the Rayo Nodes enrolled within the Rayo cluster.
  • ClientApplications: Returns information about the active Applications on this Rayo cluster.
  • ActiveClients: Returns information about the active clients (multiple clients can be connected to the same Rayo application).
  • ActiveMixers: Returns information about the active mixers in the Rayo cluster.

Operations exposed

  • getClientApplication/appId: Returns information about the given client application.
  • getResourcesForAppId/appId: Returns information about the client resources connected for a given application id.
  • getResourcesForJid/jid: Returns information about the client resources connected for a given application's jid.
  • getResourcesForAppId/appId: Shows the addresses registered for a given application's id.
  • getResourcesForJid/jid: Shows the addresses registered for a given application's jid.
  • getRayoNodes/platform: Shows information about the Rayo nodes registered for the given platform name.
  • callInfo/callId: Returns information for the given call id.
  • mixerInfo/mixerId: Returns information for the given mixer id.
  • activeVerbs/mixerId: Returns the list of active verbs for the given mixer id.

GatewayStatistics

The GatewayStatistics MBean exposes usage information for the Rayo cluster.

JMX URL: com.rayo.gateway:Type=GatewayStatistics

Attributes exposed:

  • ActiveCallsCount: Number of active calls.
  • TotalCallsCount: Total number of calls.
  • ActiveMixersCount: Number of active mixers.
  • TotalMixersCount: Total number of mixers.
  • ActiveClientsCount: Number of active Rayo applications.
  • TotalClientsCount: Total number of Rayo applications that have used this Rayo Gateway.
  • TotalResourcesCount: Total number of client resources that have used this Rayo Gateway.
  • ActiveRayoNodesCount: Number of active Rayo Nodes.
  • MessagesCount: Total number of messages processed by this Rayo Gateway.
  • ErrorsCount: Total number of errors that this Rayo Gateway has dealt with.

Admin

The Admin MBean provides an administrative interface to your Rayo Gateway and cluster.

JMX URL: com.rayo.gateway:Type=Admin,name=Admin

Attributes exposed:

  • QuiesceMode: Returns the current Quiesce status for the Rayo Gateway.
  • ServerName: Returns the Gateway's server name.
  • ClientApplications: Returns information about the active Applications on this Rayo cluster.
  • ActiveClients: Returns information about the active clients (multiple clients can be connected to the same Rayo application).
  • ActiveMixers: Returns information about the active mixers in the Rayo cluster.

Operations exposed

  • enableQuiesce: Quiesces this Rayo Gateway. The Rayo Gateway will not serve any further requests from Rayo Nodes or client applications.
  • disableQuiesce: Disables quiesce mode on this Rayo Gateway. The Rayo Gateway becomes available again to serve requests from Rayo Nodes or client applications.
  • setLogLevel/logger/logLevel: Allows you to alter the Log4j log level configuration for a specific logger at runtime. As a Logger, you can specify any package name like com.rayo, com.voxeo, com.rayo.server and so on. As level, you may use Log4j Log levels like: DEBUG, INFO, WARN, ERROR, etc.
  • blacklist/platformId/hostname/blacklisted: This method adds to a blacklist or removes from a blacklist the given hostname on the given platform. This method is normally used to disable a given Rayo Node from the cluster.
  • maxDialRetries/retries: Sets the maximum number of redial attempts that the Rayo Gateway should be try in case of failure.
  • ban/jid: Bans the given Rayo application from the cluster.
  • unban/jid: Unbans the given Rayo application from the cluster.
  • removeNode/jid: Removes a node from the cluster.
  • registerApplication/platform/name/jid: Registers an application on the cluster. It receives the name of the platform, the name of the application and the application's jid.
  • registerAddress/jid/address: Adds an address to the given application's jid.
  • unregisterApplication/jid: Unregisters the given application from the cluster.
  • unregisterAddress/address: Unregisters the given address from the cluster.
Clone this wiki locally