Making Sip Implementation using WebRTC

[kinsleykajiva]

Hi. So I have an open-source project I am making called Ripple Server. I am trying to integrate mjsip into the server as a plugin feature that users can use if they require it to be able to make calls via this SIP library.

I need help to further understand how to implement this library. Yes, I am able to make a registration and pass the SDP from a client, but I get an error after the fact, and it's very from null media description objects from the library. How can I go about fixing this? Yes, the call also gets accepted, and the status can also be in progress as well. Please, can you assist?

[haumacher]

Hi, what exactly are you trying to do? You have a client (e.g. web page) that opens a RTC stream to your Ripple Server, right? How exactly comes mjSIP into play? Do you want to forward the call to a VOIP server? Or are you about to implement a VOIP server to connect VOIP calls to WebRTC conference rooms? Can you provide sample code, what you have tried to do with mjSIP and at what stage it exactly broke?

[kinsleykajiva]

Thank you @haumacher for responding .

Here is a Ripple-PR I am working with . But not to bore you with the server setup on how to access or things around let me show the core areas on how this is set to work or my intended goal .

I do have a PBX server I am using to test , I ma using Asterisk to test all of this and have a test sip user already on Asterisk for this . I am not a VoIP engineer as much or expert but I am getting along as much .

Sip Registration is done

// file SipUserAgentPlugin.java
public SipUserAgentPlugin( CommonAbout commonAbout, final Integer thisObjectPositionAddress,
	                           String realm, String username, String displayName, String password, String host, int port ) {
		setTransaction("0");
		System.out.println("xxxxx SipUserAgentPlugin constructor realm -  " + realm);
		System.out.println("xxxxx SipUserAgentPlugin constructor username -  " + username);
		System.out.println("xxxxx SipUserAgentPlugin constructor displayName -  " + displayName);
		System.out.println("xxxxx SipUserAgentPlugin constructor host -  " + host);
		System.out.println("xxxxx SipUserAgentPlugin constructor port -  " + port);
		System.out.println("xxxxx SipUserAgentPlugin constructor password -  " + password);
		this.commonAbout = commonAbout;
		this.thisObjectPositionAddress = thisObjectPositionAddress;
		SipConfig sipConfig = new SipConfig();
		sipConfig.setHostPort(port);
		UAConfig uaConfig = new UAConfig();
		uaConfig.setAuthRealm(realm);
		uaConfig.setUser(username);
		uaConfig.setDisplayName(displayName);
		sipConfig.setViaAddr(host);
		uaConfig.setAuthPasswd(password);
		uaConfig.setRegistrar(new SipURI(host, port)); // Set the SIP server address
		SchedulerConfig schedulerConfig = new SchedulerConfig();
		MediaConfig     mediaConfig     = new MediaConfig();
		PortConfig      portConfig      = new PortConfig();
		UIConfig        uiConfig        = new UIConfig();
		sipConfig.normalize();
		uaConfig.normalize(sipConfig);
		ConfiguredScheduler configuredScheduler = new ConfiguredScheduler(schedulerConfig);
		var                 sipPr               = new SipProvider(sipConfig, configuredScheduler);
		
		init(sipPr, portConfig.createPool(), uaConfig, uiConfig, mediaConfig);
	}  

private void init( SipProvider sip_provider, PortPool portPool, UAConfig uaConfig, UIConfig uiConfig, MediaOptions mediaConfig ) {
		
		this.sip_provider = sip_provider;
		this.uaConfig = uaConfig;
		this.uiConfig = uiConfig;
		this.mediaConfig = mediaConfig;
		ua = new RegisteringUserAgent(sip_provider, portPool, this.uaConfig, this);
		//streamFactory = createStreamerFactory(this.mediaConfig, this.uaConfig);
		streamerFactory = createStreamerFactory(mediaConfig, uaConfig);
		changeStatus(UA_IDLE);
		log.info("Starting registration");
		log.info("Starting registration -- " + uaConfig.isRegister());
		if (uaConfig.isRegister()) {
			ua.loopRegister(uaConfig.getExpires(), uaConfig.getExpires() / 2, uaConfig.getKeepAliveTime());
			
			/*var rc = new RegistrationClient(sip_provider, uaConfig, new RegistrationLogger());
			if (uiConfig.doUnregisterAll) {
				rc.unregisterall();
			}
			
			if (uiConfig.doUnregister) {
				rc.unregister();
			}
			
			rc.register(uaConfig.getExpires());*/
			
		}
		
	}

So the above sample code is when the Sip user is Registred I make use of call back to validate if registration has succeded or failed per user or each attempt.

The web-browser client sends SDP details and I have a custome function I call to make an out going call . but I make the call as following

var          nameAddress           = org.mjsip.sip.address.NameAddress.parse("sip:2710210@vafey.commonresolve.co.za:9099");
SdpMessage sdpMess = new SdpMessage(sdp);
		ua.call(nameAddress, sdpMess);
		changeStatus(UA_OUTGOING_CALL);

Here are the logs or exception I end up getting

[2024-03-20 22:34:04] DEBUG: [org.mjsip.sip.dialog.Dialog]: Set state of dialog to: D_INIT
[2024-03-20 22:34:04] DEBUG: [org.mjsip.sip.dialog.InviteDialog]: inside invite(callee,caller,contact,sdp)
[2024-03-20 22:34:04] DEBUG: [org.mjsip.sip.dialog.InviteDialog]: inside invite(invite)
[2024-03-20 22:34:04] DEBUG: [org.mjsip.sip.dialog.Dialog]: Set state of dialog to: D_INVITING
[2024-03-20 22:34:04] DEBUG: [org.mjsip.sip.dialog.Dialog]: Updated dialog ID to: 656380898400@vafey.commonresolve.co.za-668770066957-null
[2024-03-20 22:34:04] DEBUG: [org.mjsip.sip.provider.SipProvider]: Adding SipProviderListener: 656380898400@vafey.commonresolve.co.za-668770066957-null
[2024-03-20 22:34:04] INFO: [org.mjsip.sip.transaction.InviteTransactionClient]: new transaction-id: 656380898400@vafey.commonresolve.co.za-1-INVITE-client-z9hG4bKe38f346e
[2024-03-20 22:34:04] DEBUG: [org.mjsip.sip.provider.SipProvider]: Adding SipProviderListener: 656380898400@vafey.commonresolve.co.za-1-INVITE-client-z9hG4bKe38f346e
[2024-03-20 22:34:04] DEBUG: [org.mjsip.sip.provider.SipProvider]: using transport udp
(22:34:04)(SipUserAgentPlugin.java:267)| state: OUTGOING_CALL
[2024-03-20 22:34:04] DEBUG: [org.mjsip.sip.provider.SipProvider]: Message passed to transaction: 656380898400@vafey.commonresolve.co.za-1-INVITE-client-z9hG4bKe38f346e
[2024-03-20 22:34:04] DEBUG: [org.mjsip.sip.provider.SipProvider]: using transport udp
[2024-03-20 22:34:04] INFO: [org.mjsip.sip.transaction.InviteTransactionClient]: new transaction-id: 656380898400@vafey.commonresolve.co.za-2-INVITE-client-z9hG4bK9c79f5f2
[2024-03-20 22:34:04] DEBUG: [org.mjsip.sip.provider.SipProvider]: Adding SipProviderListener: 656380898400@vafey.commonresolve.co.za-2-INVITE-client-z9hG4bK9c79f5f2
[2024-03-20 22:34:04] DEBUG: [org.mjsip.sip.provider.SipProvider]: using transport udp
[2024-03-20 22:34:04] DEBUG: [org.mjsip.sip.provider.SipProvider]: Message passed to transaction: 656380898400@vafey.commonresolve.co.za-2-INVITE-client-z9hG4bK9c79f5f2
[2024-03-20 22:34:04] DEBUG: [org.mjsip.sip.dialog.InviteDialog]: inside onTransProvisionalResponse(tc,mdg)
[2024-03-20 22:34:06] DEBUG: [org.mjsip.sip.provider.SipProvider]: Message passed to transaction: 656380898400@vafey.commonresolve.co.za-2-INVITE-client-z9hG4bK9c79f5f2
[2024-03-20 22:34:06] DEBUG: [org.mjsip.sip.dialog.InviteDialog]: inside onTransProvisionalResponse(tc,mdg)
[2024-03-20 22:34:06] DEBUG: [org.mjsip.sip.dialog.Dialog]: Updated dialog ID to: 656380898400@vafey.commonresolve.co.za-668770066957-b346c405-9a06-444a-83f2-29c5fe3e0bbe
[2024-03-20 22:34:06] DEBUG: [org.mjsip.sip.provider.SipProvider]: Removing SipProviderListener: 656380898400@vafey.commonresolve.co.za-668770066957-null
[2024-03-20 22:34:06] DEBUG: [org.mjsip.sip.provider.SipProvider]: Adding SipProviderListener: 656380898400@vafey.commonresolve.co.za-668770066957-b346c405-9a06-444a-83f2-29c5fe3e0bbe
Call progress
(22:34:06)(Client.java:153)| {"feature":"SIP_GATEWAY","success":true,"eventType":"sipCallProgress","message":"Call progress"}
[2024-03-20 22:34:06] DEBUG: [org.mjsip.sip.provider.SipProvider]: Message passed to transaction: 656380898400@vafey.commonresolve.co.za-2-INVITE-client-z9hG4bK9c79f5f2
[2024-03-20 22:34:06] DEBUG: [org.mjsip.sip.provider.SipProvider]: Removing SipProviderListener: 656380898400@vafey.commonresolve.co.za-2-INVITE-client-z9hG4bK9c79f5f2
[2024-03-20 22:34:06] DEBUG: [org.mjsip.sip.dialog.InviteDialog]: inside onTransSuccessResponse(tc,msg)
[2024-03-20 22:34:06] DEBUG: [org.mjsip.sip.dialog.Dialog]: Set state of dialog 656380898400@vafey.commonresolve.co.za-668770066957-b346c405-9a06-444a-83f2-29c5fe3e0bbe to: D_CALL
[2024-03-20 22:34:06] DEBUG: [org.mjsip.sip.provider.SipProvider]: Adding SipProviderListener: 656380898400@vafey.commonresolve.co.za-668770066957-b346c405-9a06-444a-83f2-29c5fe3e0bbe
[2024-03-20 22:34:06] INFO: [org.mjsip.sip.transaction.AckTransactionClient]: new transaction-id: 656380898400@vafey.commonresolve.co.za-2-INVITE-client-z9hG4bK55d87f29
[2024-03-20 22:34:06] DEBUG: [org.mjsip.sip.provider.SipProvider]: using transport udp
Call accepted
(22:34:06)(Client.java:153)| {"feature":"SIP_GATEWAY","success":true,"eventType":"sipCallAccepted","message":"Call accepted"}
[2024-03-20 22:34:06] WARN: [org.mjsip.sip.provider.SipProvider]: Error handling a new incoming message: java.lang.NullPointerException: Cannot invoke "org.mjsip.ua.MediaAgent.getCallMedia()" because "this._mediaAgent" is null
	at org.mjsip.ua.UserAgent.findMatchingMediaSpec(UserAgent.java:362)
	at org.mjsip.ua.UserAgent.buildFlowSpec(UserAgent.java:351)
	at org.mjsip.ua.UserAgent.startMediaSessions(UserAgent.java:306)
	at org.mjsip.ua.UserAgent.onCallAccepted(UserAgent.java:506)
	at org.mjsip.sip.call.Call.processDlgInviteSuccessResponse(Call.java:411)
	at org.mjsip.sip.call.Call$InviteDialogListenerAdapter.onDlgInviteSuccessResponse(Call.java:561)
	at org.mjsip.sip.dialog.InviteDialog.onTransSuccessResponse(InviteDialog.java:1154)
	at org.mjsip.sip.dialog.ExtendedInviteDialog.onTransSuccessResponse(ExtendedInviteDialog.java:358)
	at org.mjsip.sip.transaction.InviteTransactionClient.onReceivedMessage(InviteTransactionClient.java:134)
	at org.mjsip.sip.provider.SipProvider.onReceivedMessage(SipProvider.java:923)
	at org.mjsip.sip.provider.UdpTransport.processReceivedPacket(UdpTransport.java:174)
	at org.mjsip.sip.provider.UdpTransport$1.onReceivedPacket(UdpTransport.java:88)
	at org.zoolu.net.UdpProvider.run(UdpProvider.java:183)
[2024-03-20 22:34:08] DEBUG: [org.mjsip.sip.provider.SipProvider]: Removing SipProviderListener: 341847191262@vafey.commonresolve.co.za-1-REGISTER-client-z9hG4bK09f2557e
[2024-03-20 22:34:08] DEBUG: [org.mjsip.sip.provider.SipProvider]: Removing SipProviderListener: 341847191262@vafey.commonresolve.co.za-2-REGISTER-client-z9hG4bK4f62d79b

The goal is to be able to get voice audio sent to the server then relay media to client .

[haumacher]

Hi, using an Asterisk server should be a good test candidate for the mjSIP library. As you may see from the project main page, I'm not the original author of mjSIP, but have forked the project, because the original author did not respond to requests. So what version of mjSIP do you currently use, the old 1.8 or the current version from GitHub master? The line number UserAgent.java:362 from your stack trace points to a line } else { in the master version, which does not seem to be correct.

The only production use-case, I'm currently working on with this library is the PhoneBlock answering machine, where I use this library to register a SIP client to a Fritz!Box VOIP router to catch SPAM calls. In that use-case, I only need to receive calls (reacting on INVITE messages from the VOIP registrar), not making active calls. This makes it absolutely possible that I broke the call functionality during my massive refactorings done to the lib so far.

However, your use-case is absolutely valid and really should work using mjSIP. The following from your code catches my eye:

sipConfig.setViaAddr(host);
...
uaConfig.setRegistrar(new SipURI(host, port)); // Set the SIP server address

This seems to be wrong, because the VIA address is the address where the registering client is expecting messages to be sent to from the registrar server. This is typically not the same address as the address of the registrar (with the rare case where the registering client and the registrar server operate on the same machine).

However, I don't think that this is the root cause of your problem.

I think the problem is that for the outgoing call, no MediaAgent is set up. The method you called (org.mjsip.ua.UserAgent.call(NameAddress, SdpMessage)) does not set-up a MediaAgent and therefore seems not to be publically usable at all. In 2200cf7 I added the MediaAgent to the signature of the call method. This ensures that a proper MediaAgent is set up whenever a call is started. The MediaAgent is responsible for actually handling the RTP data transmission.

You could try to set up a proper MediaAgent with a DefaultStreamerFactory using an custom AudioReceiver and AudioTransmitter that handle forwarding the RTP data to your server. You can get inspiration how to handle RTP data by looking at the ToneTransmitter that simply sends a sine wave to the called party.

[kinsleykajiva]

Hi, I saw the implementation suggestions and the code changes you made I am still testing and implementing, I will give further feedback soon.