New Router Gaps
There isn't anything standing out in the performance or configuration of the device, except some of the firewall things, which seem to be bits I'm not able to find.
Here's a quick list of things I'm seeing.
I cannot get the firewall to allow inbound ping on the WAN IP. I can ping from the Internet to its routed public IP subnet, so ICMP makes it to and through the router, but something is stopping it from responding at the WAN IP. There is a "reject all" rule that I can't see is overridden, but fairly I'm much better at iptables than I am at the pfsense firewall. This is necessary for a few things, the least of which is getting my IPv6 tunnel working.
I can't seem to get the firewall to HTTP(S) out from the WAN IP. This was one of the problems I noticed when trying to work with the Firewalla. It fails every time I try a simple command, like curl -v https://jekewa.com/
, which should give the content from root of this site. Adding to the confusion, though, if I test from the router explicitly setting the WAN interface to use, as in curl -Iv --interface igc5 https://jekewa.com/
, it will return the header information from a successful request through the CDN to the root of this domain. Additionally, if I use it's LAN interface, it also succeeds, routing back through the router. Curiously, it seems to have the correct default interface in its routing table, so I'm at a loss for why it can't "just use" that for the requests. Here, in case you see something different, and slightly obfuscated, is what it says:
netstat -4rn Routing tables Internet: Destination Gateway Flags Netif Expire default 10.0.209.133 UGS igc5 10.0.209.132/30 link#6 U igc5 10.0.209.134 link#8 UHS lo0 10.0.219.128/29 link#12 U bridge0 10.0.219.129 link#8 UHS lo0 127.0.0.1 link#8 UH lo0
Having the curl command fail without specifying the interface gives me suspicion that the internal workings for its update check and some of the packages and other stuff it wants to download may be running into the same.
I can't make firewall rules that seem to bypass the defaults, although it looks like the defaults are "block unless overridden" kinds of rules. The rules I put in for allowing inbound ping don't even get ticks on their counters. I've even gone as far as disabling the firewall functionality entirely, but it still doesn't matter.
I'm going to try one more big thing tomorrow before I consider trying to replace the installed pfsense. I'm going to reset everything and do a simpler reinstall. Pick a WAN port, static address it, change its MAC, and plug it in. Then pick a LAN port, static address it, disable NAT, and plug it and everything else into a switch. I'm not sure if I complicated everything by building a LAN bridge out of the ports that aren't used for the WAN. I can configure that later if I need to.
If I can't get the pretty basic bits to behave, I might drop OPNsense on it. OPNsense is also based on FreeBSD, starting as a fork of pfsense, but it's had more updates in the last two years, and its documentation is a bit more thorough. I hesitate a little just because I don't see in its documentation where it defines those protective defaults, either, and I'm a little concerned that the box still wouldn't be able to take pings and might end up not correctly defaulting to the WAN interface for outbound traffic.
If I can't quickly get that to behave, I'll fire up a little LINUX I'm familiar with, like Ubuntu Core (because I'm lazy), with a couple static IPs, routing, firewall, and intrusion detection, like I already have running on my servers. I don't have routing on my servers currently, but have in the past. I know I can get IPv6 running in a few minutes. I don't need NAT, DHCP, or even DNS caching or forwarding. I don't even need a web GUI, but I know a couple that would probably work if I change my mind. And I know where everything is and how it works. I'd gamble that it'd take me longer to download and install the OS than it will to get the router working.