Small DNS Snafu Blocks Mail
I recently rearranged my IP addresses after an outage on one of my ISPs. Just about everything worked well, but yesterrday it was noticed that some e-mail had been bouncing.
A short investigation found that the TLS and SSL SMTP ports weren't working for authorizing users to relay mail through my server. At first I blamed the new ISB, who had been blocking the outbound SMTP from my network (they have an option to disable that block on static IPs, which I eventually got working). I was close; the ports weren't blocked, but instead the ISP being used was my IPv6 tunnel.
I had configured some of my DNS entries with my servers' IPv6 addresses. This was fine when the IPv6 tunnel worked on the old ISP, but the new ISP somehow doesn't allow those packets through. While chatting with their support guys they had a hard time recognizing that it isn't a TCP or UDP packet, but an IPv6 packet (protocol 41) in an IPv4 envelope... I don't think they're maliciously filtering, but probably have opened TCP and UDP, but blocked some or all of the other protocols. I digress...
I made some quck DNS changes, removing the IPv6 addresses from my zone, and waited for propagation.
While I waited, I found a site to test my DNS configuration from the outside (it's all valid, according to the tool I use to maintain it...). From there I discovered that DNS does indeed use TCP (which I'd been firewalling off...fixed), and that the service I was using for secondary DNS wasn't responding. I found and configured another service, and through their tool found and corrected a couple other things in my configuration.
By the time I finished fixing my registrar entries to point to the new secondary DNS, the previously blocked e-mail started making it through. No real changes on my server (I did fix the DNS and tweak the firewall to allow TCP port 53 through), and traffic started flowing again.
I'm not sure what web or mail traffic might have been missed these few days. Mail will usually queue and wait for a few days, so that might come flooding in. Web traffic, though, is usually real-time only, so those people or services were probably disappointed. However, it seems that most browsers will try an IPv6 and then revert to IPv4 if the v6 failed, so there's that probability that people weren't impacted.
Small silver lining to this is that I had been more lax in my IPv6 firewalling, and now there's no IPv6 traffic hitting the server. I'm pretty stringent on my IPv4 firewalls, cutting off some abusive countries from my SSH and mail servers, and some also from web servers. Apologies to those in China or Russia or even India that want to read my blog posts and find their access cut off; too many other attempted abuses have led me to this drastic measure.
Although, now that I think of it...I now have fail2ban configured, so maybe I can reconfiugre the firewall to be more lenient until an abuse is discovered. I think I'd want to make fail2ban create permanent entries for somethings, like obvious fishing attempts at SSH and try to separate SMTP from HTTP abuse. I could see that being a huge rabbit hole. For now I think I'll just cut off the world I don't know.