I wasn't thinking and postponed getting my VPN working before I went somewhere I might need to use it.

I was able to find a few workarounds and solve a couple problems, so I was able to do what I needed to do remotely.

I had to make a couple tweaks to a website. Easy, done. Then I wanted to publish the changes to the container server. Usually easy, not so easy if you're not on the LAN somehow. So I set to getting things going.

First, I have been able to connect to the VPN server. And from that connection, I can ping some things, and when I checked the IP, it is hitting the Internet through the VPN instead of the network I was on. All of that was kind of tapped out in the other post. While connected, I couldn't SSH to the servers I wanted to, nor to the router through which I was connecting to the VPN. I did find I was able to SSH to my egress router. SSH is available to all nodes on my subnet, but is otherwise blocked at the egress router, since I thought I had the VPN working.

There's a router at my ISP gateway, and a small subnet where my servers with public IPs are, including the VPN-providing WiFi router. I'd configured the egress firewall to allow VPN traffic through to the WiFi router, and that has worked. I could SSH to that router, and then from that SSH session, SSH back into my LAN by SSH through one of the public servers, and since it's connected to the LAN, I was able to SSH to other servers.  It's a bit of a zig-zag, which makes me think of all the "bouncing everywhere" that is seen in the movies and crime TV shows.

One thing I found poking around on the servers was that the VPN subnet is not in the trusted list I have in my firewall rules. I added it, and was able to try connect to the SSH server where I made that change, but the SSH wouldn't connect. I found some articles indicating the MTU may be wrong, and the key exchange it was trying to use might not be making it through the VPN config. I couldn't change the MTU remotely (I did try), but I was able to get past the key exchange problem, which did allow my SSH attempts to work with the firewall changes!

Alas, it was a short victory, as many actions would freeze the SSH session. Simple things like cat or vi or even just starting screen wouldn't finish, sometimes putting bits in the terminal, sometimes just nothing. More research poked at the MTU again, and also suggested using TCP instead of UDP, which was the default I left alone when configuring the VPN server. Unable to change those, I mustered through and was able to get a couple things going with that server, so I didn't have to hop three times to get there.

On the other server, the one that I connected to from the egress router, I was able to discover that its multihome configuration was likely to blame. When asking it to ping the VPN node, it would fail. Checking the routing rules, it was trying to send that traffic through the WAN connection, which is the default Internet connection. I added a static route and was able to get past the SSH issues, and was able to stop double-hopping to reach that server!

Again, the connections were volatile, and would freeze occasionally, but baby steps.

I tweaked the MTU settings in my client config, as suggested by an article, and it made the SSH a bit more stable, but not quite 100%. I was able to reach web servers on the LAN that aren't Internet accessible, so that was a win!

Armed with the research, after I returned home, I reconfigured the VPN to use TCP instead of UDP, and changed its configuration to forward DNS settings through the VPN connection. I didn't mention that, but that seemed to be a problem when trying to connect to some web bits, is that since the VPN wasn't sharing DNS settings, the network configuration otherwise on the system was used. I also took a moment to put some host names in the dnsmasq configuration on the VPN server, so when nodes on the LAN or VPN ask for hosts, they'll get the LAN IP. This is helpful for the nodes that have public IPs also, and really helpful for nodes that aren't advertising FQDN. I also tweaked the firewall rules on the egress server to allow both UDP and TCP connections to the VPN server on the VPN port.

I downloaded my changed client configuration. I removed all the old client configuration from my two VPN clients I was trying to use, and added the new one. I connected both of them, in separate tests, to my phone's hotspot, and confirmed they couldn't reach my LAN nodes. I connected the new VPN and tried the SSH and web connections that gave me trouble before. Everything worked! I didn't change any MTU settings, and I didn't tinker with any DNS (although the one machine I was using is normally on my LAN, so it has some /etc/host entries for the LAN nodes; later I'll test removing them to see if the LAN/WiFi router dnsmasq works.

I completed the tasks that I was trying to do part way through all of that, some while I was away with the tweaks. I ran through the same steps while connected through the new TCP VPN settings, and didn't have any issues. No SSH freezes, no website complaints, and a ton of happy joy at finally getting everything working. Now if I need to do things on my LAN while I'm away, I can!