[QUESTION] Multiple incoming calls at one phone number at the same time

[HyxerPyth]

I'm working on application where I need to handle multiple incoming calls and conversations at the same time on one phone number.
I'm building an app where you can talk to an AI. So, im streaming the audio from twilio in order to transcribe text and than I use AI to respond back to customer. But I don't understand how to have multiple conversation with customers on the same phone number at the same time, without any lines and redirections to other phone numbers.

[charliesantos]

@HyxerPyth thanks for reaching out! To provide you the best answer, can I get more details about your setup? Do you have any diagrams that shows what Twilio components you're using and how they relate to each other?

[HyxerPyth]

So, 1. I use ngrok to set up a websocket link. 2. I run python server using quart it uses ngrok link and make it webscoket link.
3. I stream sound from twilio through that websocket link to transcribe it using vosk model. 4. When text is transcribed I use elevenlabs to generate the sound. 5. Once sound is generated I use this function to send audio back to twilio, it has stremSid an an argument. async def send_audio(websocket, stream_sid, audio_data):
if isinstance(audio_data, str):
audio_data = audio_data.encode('utf-8')

# Create the payload to send to Twilio
payload = {
    "event": "media",
    "media": {
        "payload": base64.b64encode(audio_data).decode('utf-8')
    },
    "streamSid": stream_sid
}

# Send the payload over the WebSocket connection
await websocket.send(json.dumps(payload))

Also, if it helps, I try to check streamSid on each step where Im getting the audio from twilio, transcribing text, generating the audio for specific streamSid and than use it to send the audio back.

That's how my app work. lmk if you need more details.

Also the websocket link is always the same for all users that are calling, but streamSid is different.

[HyxerPyth]

The app works perfectly with one user. But, when second caller connects it just keeps sending the audio to the wrong person sometimes

[charliesantos]

@HyxerPyth thanks for the details. When you say to and from twilio, are you referring to this twilio voice javascript sdk, or is twilio backend?

[HyxerPyth]

I mean regular twilio phone calls.

[charliesantos]

@HyxerPyth are you using Twilio's Media Streams product https://www.twilio.com/docs/voice/media-streams?
Or are you using the JavaScript SDK API call.getRemoteStream()?

[HyxerPyth]

Hi, I'm using Twilio's Media Streams

[charliesantos]

Thanks @HyxerPyth . In this case, can you please submit a support ticket using your Twilio account? Our support team should be able to route you to the correct team.

[HyxerPyth]

Got it! Thank you for the response!