The Ubiquiti Diaries: A Site-to-Site VPN Storyby Ganesh T S on December 21, 2022 8:00 AM EST
- Posted in
- Ubiquiti Networks
Ubiquiti Networks is a popular vendor of networking-related equipment in the SMB / SME space. Their gear is immensely popular among prosumers too, thanks to the combination of ease of use and the ability to customize for specific requirements. I have been running an Ubiquiti UniFi installation at home for the last five years or so, and recently had the opportunity to create a new deployment in another country. There were two main reasons to go with Ubiquiti for the new location - a single management plane for both sites, and the ability to easily create a site-to-site VPN.
The new installation was fairly smooth and the site-to-site VPN was up and running in a stable manner until the ISP at the remote site moved the gateway from a public-facing WAN IP to one behind a carrier-grade NAT (CGNAT). That started a deeper investigation into various options available for site-to-site VPNs with Ubiquiti's gear for different scenarios. In this process, I ended up encountering a host of issues worthy of documentation to help folks who might encounter them in their own installations. This article provides a recount of my trip down the rabbit hole - including a step-by-step guide detailing my attempts to work around the various pitfalls.
Ubiquiti Networks offers a range of products targeting the networking market. While wireless ISPs are a key market segment for the company (serviced by the airFiber line), today's piece is focused on their UniFi product line - a range of managed software-defined networking equipment for SMBs, SMEs, and prosumers. There are a number of reasons for UniFi's popularity products among tech-savvy consumers. The company had a first-mover advantage in offering a cost-effective managed SDN solution. Isolating functionality into different devices (security gateways, routers, switches, and wireless access points) allowed users to pick and choose different equipment based on their custom needs. The unified management plane for all the UniFi products enables easy maintenance while retaining deployment flexibility. Network scaling in response to requirement changes is also straightforward. The company started out with a local management controller, which has now been augmented with a cloud-based offering.
My first brush with Ubiquiti was their mFi product line (which has since been unfortunately EOL-ed). Their lineup of network-connected power outlets with energy and power monitoring, as well as remote relay control was (and continues to be) more flexible than anything else in the market - and this was without even taking the low pricing into account. I had purchased a few of their units for my home / AnandTech testing lab use, and written a short review after a couple of months of use (those units are still in deployment).
After I published the mFi review, Ubiquiti's PR department approached me with an offer to review their UniFi product line. Around that time back in 2017, I had the opportunity to lay out a wired Cat 6 backbone for all the rooms in my house here in California. I took up the offer to spec out a UniFi system for testing out. The USG Pro 4 gateway took up the routing duties with a UniFi Cloud Key (first generation) performing controller duties. Access points with varying capabilities were mounted around the house to avoid wireless dead-spots. A number of switches were placed in the media center and different lab locations. I ended up augmenting the system with additional PoE switches and in-wall APs on my own.
The system was configured with the usual guest wireless network, and a bunch of different VLANs (serving the IoT devices in the house, the home lab equipment, and another for devices such as the common family desktop, phones, etc.). On the whole, it was an overkill for a residential installation. That said, the deployment has held its own over five years of stressful usage (and still going strong). The only hiccup I had was when the CloudKey controller became inaccessible on the network a couple of years back. It turned out that a power interruption had ended up corrupting the database - nothing that a few SSH commands (thanks to the helpful community) couldn't resolve. Since then, I ended up investing in a UPS for the rack holding the UniFi equipment to avoid the recurrence of such scenarios.
Such issues are also the reason why I recommend Ubiquiti equipment only to tech-savvy users. In almost all cases, calling up the company's support line and creating a ticket ends up being a waste of time. There are innumerable resources online (both the company's own users forum, as well as countless prosumer bloggers such as Scott Hanselman and Troy Hunt. In light of reviews from such sources, there is not much for readers to gain from posting yet another review of the Ubiquiti UniFi lineup. Instead, I am hoping to take up specific use-cases and figure out how Ubiquiti's product lineup can address those in these series of articles.
Earlier this year, my parents back in India decided to downsize their home. I took the opportunity to revamp their home network from the ground-up. I had been intending to add features to the home network of my parents, but had never had the opportunity because my visits were becoming infrequent. However, with my first visit post-pandemic, I wanted to get a few things set up as part of their move:
- Easier remote management and troubleshooting of network issues without the need for port forwarding.
- Ability to seamlessly use their Indian home network during travel / visits over here to California
- Ability to perform secure remote offsite backups for my data without relying on an external cloud storage provider
- Ability to seamlessly utilize Indian OTT service subscriptions irrespective of user location either in California or in India
When I initially set up the Cloud Key back in 2017, there was no requirement to use a cloud account. Unfortunately, the UniFi Network mobile application user experience became quite onerous without a ui.com ID a couple of years back. I caved in and ended up associating my installation with a cloud ID just for this purpose. Since I was already managing my network through this ID, it became a straightforward decision to go with Ubiquiti for the deployment back in India.
The key to fulfilling the above requirements was a secure VPN tunnel between my home network here in California and my parents' network in India. Prior to traveling, I arranged for a Ubiquiti Dream Machine to be delivered to the new home. The Ubiquiti UniFi Dream Machine is an all-in-one solution / UniFi starter kit. It integrates a 4-port switch, a 4x4 802.11ac access point, a security gateway, and an integrated controller. The Annapurna Labs AL314-based solution comes with a single WAN port, and is an acceptable solution for most home networks in the the 1000 sq. ft - 1200 sq. ft range.
From my use-case perspective, I wanted a solution that would support simple VPN tunnel configuration and easy app-based access for both the US and Indian networks via a single interface.
The Evolution of UniFi - A Short Recap
Ubiquiti's UniFi lineup was launched after their lineup of edge-focused products for WISPs started gaining traction in other markets. These EdgeRouters and EdgeSwitches were based on Vyatta OS, and the UniFi products initially started out with the same EdgeOS firmware base. The UniFi Security Gateway Pro 4 in my primary deployment runs EdgeOS to date.
The USG Pro 4 is based on Cavium's OCTEON II networking SoC, with a MIPS64 application processor. However, Ubiquiti's latest gateways / routers / switches in the UniFi lineup now run a custom Debian-based Linux distribution. The UniFi Dream Machine uses the Annapurna Labs AL314, and runs a distribution meant for the AArch64 platform. The UniFi OS itself runs as a container using podman.
The end result is that there are quite a number of disconnects between the features available on EdgeOS and UbiOS / UniFi OS. Migration from the EdgeOS line to UniFi OS is not straightforward enough for heavily customized installs. With focus shifting to UbiOS / UniFi OS, the updates for the older equipment have become few and far apart. While that might not be a concern for stable networks, it has unfortunately not kept up to date with evolving network security practices. For example, Android's recent releases have completely dropped support for L2TP VPNs, while EdgeOS has L2TP as the recommended VPN server type. This brings us to the topic of VPNs.
VPN Server Options in Ubiquiti's Stack
Ubiquiti offers a range of VPN options depending on the gateway being used. At home here in California with the USG Pro 4, I have been running a L2TP VPN server (allowing me to connect to it from public coffee shops and airports for secure browsing purposes) for several years now. I had minimal trouble setting it up for access from a Windows notebook. However, as mentioned in the previous sub-section, this VPN server is of no use for my mobile phone running Android 12. The USG Pro 4 also supports PPTP VPN, but it is not recommended even by Ubiquiti themselves.
The primary option for a VPN server in the UniFi Dream Machine running UbiOS / UniFi OS is quite different.
Here, Teleport (Ubiquiti's customized Wireguard implementation) takes precedence. This is a one-click VPN more in tune with today's mobile-first ecosystem. Clients are authorized via invites that can be generated either from the configuration page (on the unifi.ui.com cloud, or via the machine's local IP) or the UniFi Network mobile app. The invites can be opened on the client device using the Wifiman mobile application. The unfortunate aspect here is that Windows users are out of luck. While MacOS, Android, and iOS are covered, Windows users are left in the lurch. This is a hugely disappointing situation given that the L2TP option in EdgeOS works with Windows clients, but not Android and the Teleport option in UbiOS / UniFi OS works with Android clients, but not Windows. It must be noted that the UDM still supports L2TP for Windows clients.
Under the Teleport & VPN section, Ubiquiti also provides an option to create site-to-site VPNs, which is where our story starts.
Post Your CommentPlease log in or sign up to comment.
View All Comments
Notmyusualid - Friday, December 23, 2022 - linkRidiculous comment.
IPv6 is the bomb.
Just because you have to learn something new, doesn't detract from its usefulness.
ballsystemlord - Saturday, December 24, 2022 - linkRight! In facing the challenge to post on AT, I learned how to read through my messages to check them for spelling mistakes because there is no edit button. ;)
Skeptical123 - Thursday, December 22, 2022 - linkSure, let's add another layer of abstraction/complexity to the Internet protocol suite (TCP/IP stack) . That's a great idea. ipv4 and ipv6 is not something one expects nor do most consumer deal with. They are however one of the more if not the simplest bit of said suite/stack.
lalagon - Wednesday, December 21, 2022 - linkUpdating the firmware on a ubiquiti product it's like trying a lottery ticket...
at_clucks - Thursday, December 22, 2022 - linkOh you can say that again... I only update when I can afford to lose the connectivity :D.
Samus - Thursday, December 22, 2022 - linkEspecially if you don't have the controller and java updated and the hardware adopted in the database. It's overcomplicated to the point I wonder why they set out to find a solution looking for a problem that didn't exist.
Seraphimcaduto - Sunday, December 25, 2022 - linkWow I thought it was just me being overly dramatic about the update process on my dream machine, glad I’m not the only one that thinks this. I love the uptime on my dream machine but the update process feels like a jump to light speed without a nav computer.
jhoff80 - Wednesday, December 21, 2022 - linkFor what it's worth, 'plain' Wireguard is supposedly available on the Dream Router, Dream Wall, and Dream Machine Pro SE (anything with OS 3.0). My regular Dream Machine Pro is still awaiting that update though, so I can't say for sure how well it works.
Maltz - Wednesday, December 21, 2022 - linkI use UniFi switches and AP's, but I wouldn't touch their routers except for only the most basic consumer-level stuff that for some reason required remote management. (I run a USG at my mom's house.) Their camera systems are good, too, but they have a habit of pulling the rug out from existing users in their non-network product lines like phones and NVRs.
Seraphimcaduto - Sunday, December 25, 2022 - linkI have their switches, APs and the Dream Machine and you’re right; the APs and switches are awesome but the router has some strange limitations. The uptime has been a lot better than most of the other consumer gear I’ve had though. if anyone has any better suggestion for a router, I’m all ears.