DB Server Hiccuped
Maybe it was the unplugged Ethernet cable.
This web server and a couple other things that depend on the database server stopped working because the database stopped responding. I was unable to connect to the database, nor could I SSH to the server running the database, nor its management port. It's my happy, hardy Sun Fire T5220, that never gives me trouble, so I'm a little put out that it did this time.
After a few trips between desktop and where the servers are (grumbling more than once that I should have grabbed a tablet or laptop and just work from near the servers, or that I should find the power adapter for my KVM so I can just use one of the computers with a display), I finally got it sorted.
Part way through, I powered off the server to let it just reboot. It usually does that fine by itself. I couldn't see the power button in the front in the dim night light, so I pulled the PSU cables and waited for its lights to go out. After plugging them back in, the fans kicked in and the LEDs started flashing. I returned to the desktop, but still couldn't connect to the management port. After pressing the power button (found it) and returning for one more trek, I looked at the back and saw the Ethernet cable was pulled from the management port. No good reason for that!
I plugged it back in, and was able to connect to the management port. From the management port, I was able to connect to the system console, and from there watch it finish booting. It took a lot longer than I wanted it to for the RAM to be checked and the system to boot. Once booted, I started the zone with the database on it, which seemed to take a while, too. But, a handful of minutes after finally getting the console running, the databases are back, and the web server and other things are all performing as expected.
I'll probably try to figure out why it hiccuped, but honestly the thing ran for more than a year without a restart. It's probably an old enough version to consider replacing. It's a good little server, though, with really only one job, a ton of RAM, a healthy amount of redundant storage, and only draws about 100W of power under normal use...it's hard to think of turning it off.