Guessing something got overloaded down the wires, probably because of all of the air conditioners running on this hot, hot day.

Just getting into the baseball game when everything went dark. Well, the lights and screens all turned off, but it wasn't really dark because it's midday.

I checking out the windows, I could see lights in the neighbors' across the street, but not to either side or across the alley. The lack of lights is a little inconclusive because it's daytime. One neighbor across the alley has a string of lights that's almost always on around their chicken fence, and that was off.

I checked with the fam to make sure everyone was aware. Mostly to make sure the kids didn't start opening the fridge, as they're wont to do, or try to shower and expect a hair dryer to work. After a brief chat with the missus, where she confirmed she'd reported the outage to the electric company already, but they didn't have an estimate yet.

I wandered to the data center to find that two of the UPSs were off already, and one was about half drained. One of the UPSs is having a known battery problem. I actually planned to swap it today as it was complaining about a failing battery after the last power drop, but the battery had a full charge, so I let it go. Since the power is out, I planned to dig into that.

The kids and missus took this as a great time to get out of the house, since the AC is off, and the pool at the park seems much more inviting. It took a bit to get everyone ready, but they finally were. While waiting, the wife reported that she'd heard from the electric company, and they estimated a return in about 90 minutes. Once they all got ready, we paraded to the garage. I pulled the lever on the garage door, lifted it to let them out, and engaged the lever again. The kids were impressed, but really I'm tall enough to reach the rope on the lever, and the heavy door is assisted with springs, but I didn't share all that. After they drove off, I thought to do a quick grab of a beer and sit on the deck in the quiet (especially without the din of HVAC compressors from all the houses), but thought it might take me too long to dig into the back where the beer always gets pushed.

I went into the basement to dig into the UPS, and realized (again) how dark it is in the data center corner. It would be potentially dangerous to try to wrestle the UPS from the rack, and its unfortunate "cable management." I decided to wait for the power to return.

The power returned after only about an hour. I pulled the UPS, rearranging a little of the cable spaghetti in order to remove it, and replaced the battery. I was momentarily disturbed as I couldn't plug one of the battery connectors in, thinking I might have gotten a near-miss to the right replacement battery. I compared it to the old one and realized there was just a little plastic cover on the one connector, to ensure you're purposeful with plugging it in, I guess. Battery installed, I pushed the power button. It sprang to life with an alert that it was unplugged, but a little bit of power in the battery. I plugged it back in, to silence that occasional beeping, and slid it back into place. I plugged the equipment into it, and made sure all the machines were on.

I returned to the home office and shelled into the various servers and turned them on, ran the post-boot firewall fixing scripts, and did a quick scan of the necessary services. Logs flowed and everything seems good. I finished with this blog post as as a final test.