Consider briefly this PBX network design:
Originally, google voice was connected to the cloud server hosted at digital ocean. Things ran great for a while, but I didn't like having that server being the point of connection with google voice. The only reasons gVoice is integrated to the network is to provide simultaneous ring with my desk phone and to allow my desk phone to place calls out through google voice. Asterisk doesn't take the calls from google voice unless they are answered by the desk phone; google voice is still responsible for simultaneous ring with my cellphone as well as providing voicemail service.
I run asterisk on the digital ocean host to provide resiliency for times when my home network is not available. Since I don't need this resiliency on my google voice line, it made sense to connect google voice directly to my local gateway in order to reduce potential lag.
When I made this change, I suffered one-way audio troubles. Twice! This post outlines some of what I found and what I tried in the course of investigating and repairing these troubles.
It wasn't the network
When contemplating "no audio" issues on VoIP lines, I first consider the firewall and ports required by the audio streams. I know a lot about networking, configs, and making these systems run, but I don't know terribly much about the inner-workings of VoIP protocols, including XMPP. Importantly, though, it appears google voice uses RTP for audio paths. These ports were already open for inbound traffic on my gateway. I tested this by running
ping -c1 voice.google.com tcpdump -i eth0 host [IP of voice.google.com]
Then I made a call. This gave me replies like the following, which have been mangled to improve legibility:
15:24:03.736808 IP me.16476 > google.19305: UDP, length 56 15:24:03.769998 IP google.19305 > me.16476: UDP, length 68 414 packets captured 414 packets received by filter 0 packets dropped by kernel
I only know so much about reading tcpdumps, but deduced two important things from looking at this information.
- Google is communicating with my host using a random UDP port such as 16476, which is within the expected RTP port range defined in
- 0 packets were dropped by kernel, meaning the firewall isn't at fault
"Port unreachable" doesn't matter
In the tcpdumps, I also tended to see things like this:
google.19305 > me.16476: UDP, length 172 me > google: ICMP me udp port 16476 unreachable, length 208
I thought: well this must be the problem! It looks like traffic in one direction is getting rejected....
I had no idea what caused this (still don't), but turns out these messages didn't matter. With working voice channels, I still see stuff like this. So while I don't know what these indicate, I don't think they're related to one-way audio trouble and they might not indicate any trouble at all.
Inbound calls were fine
Both my one-way audio troubles only affected outgoing calls. I found it curious that when I received calls from google voice, everything would work. But first I had a problem where callers couldn't hear me, and then I had a problem where I couldn't hear them. I performed a lot of tests through the course of investigating this trouble. The troubles were consistent on outbound calls, and the destination (phone number beyond google voice) didn't matter.
Spoiler: looking back from issue#2, this seems to make sense if the codec conversion was a factor. I don't know how asterisk handles codec negotiations, but if an inbound call from google voice were requesting ulaw, and my SIP station just said 'sure, I can do ulaw,' that could work. But if on an outbound call my station requested gsm, which gVoice rejected and thus forced asterisk to convert between ulaw and gsm, perhaps that's what caused the trouble to only affect outbound calls. But then why would the audio go one-way in one direction, and then go one-way in the other direction? Excellent question! I don't know! But below is what I experienced, and what solutions I found.
One-way audio issue #1
The first problem I had was when I placed calls, I could hear callers but they could not hear me. I looked into a lot of things and asked the Internet a lot of questions, but failed to pinpoint the cause. When I turned up the verbosity in asterisk, I could see this during calls:
-- Motiffirstname.lastname@example.org answered SIP/107-003 -- Channel SIP/107-003 joined 'simple_bridge' basic-bridge <036a556e> -- Channel Motifemail@example.com joined 'simple_bridge' basic-bridge <036a556e> > Bridge 036a556e: switching from simple_bridge technology to native_rtp > 0x7ffbc8016ec0 -- Probation passed - setting RTP source address to [gvoice]:19305 > 0x7ffbf4011210 -- Probation passed - setting RTP source address to [sipclient]:11798 -- Channel SIP/107-003 left 'native_rtp' basic-bridge <036a556e>
The timing of these messages coincided with when I would experience audio path problems. It looks like two things are happening here.
- asterisk is handing off the audio paths to the RTP ports
- it's also switching from simple_bridge to native_rtp
Handing off communication to RTP ports seems easy enough to understand, even with my limited knowledge of these protocols. But switching from simple_bridge to native_rtp? That sounds a lot deeper into the engineering of asterisk than I'd like to go.
I looked around but didn't find a good explination. What I did find told me to turn off canreinvite in
sip.conf. So I did.
After publishing this change, my problem was fixed! I made a test call, blew into the handset and heard it come out the other end! Callers could hear me now and my one-way audio problem was fixed.
I tried to get back to business
.....which brought me to.....
one-way audio issue #2
Fixing one problem presented another. Now callers could hear me, but I couldn't hear them! I didn't recognize this at first because google voice doesn't provide ringback. Look at the dial command
r at the end means asterisk is providing the ringback, which is necessary because google voice doesn't. So I would place a call, hear it ring, but then when the caller picked up I only got dead air.
Aside: if you've been paying attention, you might be asking yourself: did he really have one-way audio before, or when he said he could hear callers, was he talking about ringback? Yes, it was one-way audio in the other direction before. It wasn't until I got dead-air after ringback that I realized asterisk was providing the ringback.
Luckily, though, it seems I solved this problem before I even really started looking into it. I thought: well, if the other problem was an asterisk internal bridge issue of some sort, what about the codec? That's probably some other asterisk internal bridge, right? According to the guide, google voice prefers ulaw. But my sip set was configured to prefer gsm. Not for any reason in particular, but because the quality seems fine and because asterisk seems to prefer gsm since it's the default format of their audio files. Let's reconcile the codec; time to edit
 disallow=all allow=ulaw allow=gsm
The above version is fixed; I reordered ulaw and gsm to put ulaw first. After committing the change I placed a call. It worked! Awesome. Now I have two-way audio inbound and outbound. My google voice works again, and now it's connected more directly to the sets that use this service.
All this has made me wonder about how the audio path is established between the two parties. It seems once the call gets established, the audio channels are handed off for direct communication between the SIP station and google voice, using RTP, and in a sense bypassing asterisk. Well does that mean I had a NAT problem and my solutions are only masquerades disguising a simpler truth? I don't think so. My asterisk servers have public IPs, which connect directly to my providers (including google voice). And while my SIP stations may have private IPs, there's no NAT between them and asterisk. I'll have to study this in more detail at some point, but for now it's all fixed. Edit: Turns out I was right to wonder about this, and found the anser; see the next section!
OK, if you've read this whole post, you deserve a gold star for endurance. You didn't have the exact same problems, did you? It seems like I found a couple unlikely config confluences that I really don't expect are widespread. But if you did read this whole post, than you might be curious, like I was, as to whether or not issue#1 was a real issue, or if it was really codec all along.
So with my codecs resolved, I went back and re-enabled canreinvite. Sure enough, I went back to the one-way audio trouble explained in issue#1 -- inbound audio only, and only affecting outbound calls. Damnedest thing. So what does canreinvite actually do? Or perhaps a better question: what's it supposed to do?
I found this explination:
If both phones support the 'canreinvite' option, they setup direct RTP connections with each other. Otherwise they set up RTP connections with Asterisk, which bridges the two channels, translating the encoding if necessary
Well that explains it! Yes, canreinvite=no. For a phone that is behind a firewall, enabling canreinvite sounds like I'd be asking for NAT trouble. Much better; now I've learned something!