The Ubiquiti Diaries: A Site-to-Site VPN Story
by Ganesh T S on December 21, 2022 8:00 AM EST- Posted in
- Networking
- Ubiquiti Networks
- UniFi
- VPN
The CGNAT Spanner in the Works
Carrier-Grade NAT (CGNAT) is used by ISPs to circumvent the lack of IPv4 addresses available for allocation to their customers. As more and more devices come online and connect to the Internet directly, it is not surprising that there is a dearth of IPv4 addresses to hand out. CGNAT is mostly encountered with WAN connections enabled by mobile service providers (think of smartphones connecting to a 4G or 5G network). However, many wired ISPs (such as Airtel in India) have also started rolling out CGNAT. The easiest way to identify if one is behind a CGNAT is to search for one's IP address on Google. Google helpfully displays your public IP address as the first result. If this address is different from the WAN IP reported by your gateway (and that is usually between 100.64.0.0/16 and 100.127.0.0/16), you are likely behind a CGNAT. This is different from a local IP gathered by a gateway placed behind ISP-supplied equipment that acts as a DHCP server itself.
CGNAT is rarely a problem for regular web browsing and content consumption. Problems start cropping up when dealing with services that require opening up ports on the gateway for bidirectional communication - such as VoIP services, multi-player gaming, and VPN setups. Even simple things taken for granted like systems in the network being able to communicate with a remote NTP server (for time synchronization) are not possible. Since the NAT happens at the ISP end, it is not possible for port forwarding to be configured either. Some services manage to find their way around CGNAT using STUN / TURN, but power users end up finding one roadblock after another. On the whole, CGNAT is a pain in the neck for users accustomed to obtaining public-facing WAN IPs.
NTP Synchronization Attempt behind CGNAT, and the 'Solution' (Using a 'Local' NTP Server)
Ideally speaking, configuring the Site-to-Site Manual IPSec VPN on the USG Pro 4 (having a public WAN IP) with a remote server address of 0.0.0.0, and providing the USG Pro 4's WAN IP as the remote server address on the UDM side (behind the CGNAT) should have worked by allowing the CGNAT side to initiate the site-to-site connection with the server. I did try out this suggestion made by an Ubiquiti employee many years ago - however, the initial key exchange / handshake process appeared to be break down with the response from the USG Pro 4 apparently getting dropped by the UDM's ISP.
An online search for enabling site-to-site VPNs with one site behind a CGNAT brought up two sets of results - one suggesting a connection via an intermediate VPS (cloud server hosted by a 3rd party), and the other trying to make the public WAN IP-possessing side to act as an OpenVPN server and the CGNAT side to act as a client and initiate the connection. I was not interested in bringing a VPS into the picture, and my trials with OpenVPN did not yield any promising results.
At this point, I realized that my Android phone was able to open up a Teleport connection with the UDM despite the gateway going behind CGNAT. So, my first attempt was to get a system to connect to the Teleport VPN and route traffic from specific devices through the system's Teleport VPN connection.
35 Comments
View All Comments
Notmyusualid - Friday, December 23, 2022 - link
Ridiculous comment.IPv6 is the bomb.
Just because you have to learn something new, doesn't detract from its usefulness.
ballsystemlord - Saturday, December 24, 2022 - link
Right! In facing the challenge to post on AT, I learned how to read through my messages to check them for spelling mistakes because there is no edit button. ;)Skeptical123 - Thursday, December 22, 2022 - link
Sure, let's add another layer of abstraction/complexity to the Internet protocol suite (TCP/IP stack) . That's a great idea. ipv4 and ipv6 is not something one expects nor do most consumer deal with. They are however one of the more if not the simplest bit of said suite/stack.lalagon - Wednesday, December 21, 2022 - link
Updating the firmware on a ubiquiti product it's like trying a lottery ticket...at_clucks - Thursday, December 22, 2022 - link
Oh you can say that again... I only update when I can afford to lose the connectivity :D.Samus - Thursday, December 22, 2022 - link
Especially if you don't have the controller and java updated and the hardware adopted in the database. It's overcomplicated to the point I wonder why they set out to find a solution looking for a problem that didn't exist.Seraphimcaduto - Sunday, December 25, 2022 - link
Wow I thought it was just me being overly dramatic about the update process on my dream machine, glad I’m not the only one that thinks this. I love the uptime on my dream machine but the update process feels like a jump to light speed without a nav computer.jhoff80 - Wednesday, December 21, 2022 - link
For what it's worth, 'plain' Wireguard is supposedly available on the Dream Router, Dream Wall, and Dream Machine Pro SE (anything with OS 3.0). My regular Dream Machine Pro is still awaiting that update though, so I can't say for sure how well it works.Maltz - Wednesday, December 21, 2022 - link
I use UniFi switches and AP's, but I wouldn't touch their routers except for only the most basic consumer-level stuff that for some reason required remote management. (I run a USG at my mom's house.) Their camera systems are good, too, but they have a habit of pulling the rug out from existing users in their non-network product lines like phones and NVRs.Seraphimcaduto - Sunday, December 25, 2022 - link
I have their switches, APs and the Dream Machine and you’re right; the APs and switches are awesome but the router has some strange limitations. The uptime has been a lot better than most of the other consumer gear I’ve had though. if anyone has any better suggestion for a router, I’m all ears.