Well… this has been an interesting day so far. First, at 12:57 AM we’re rudely awakened by a really loud splash in the pool. Of course this got the cats all excited, so I got up and went outside to see what all the commotion was about. Turned out to be a big raccoon that decided he wanted to take a swim. LOL After he climbs off over the fence I just went back to bed thinking all will be fine.
At 2:30 AM… I just couldn’t sleep anymore so decided to just get on up and get to work on the usual morning routine, no big deal. After getting logged into everything on the home network I see Nagios is telling me that the T-1 at the office is down but since we tend to have so much trouble with it I didn’t give it much thought. I was still able to establish a VPN connection into the corporate network over the wireless connection at the office. So again… didn’t really worry too much about the T-1. I just figured I’d power-cycle the AdTran as soon as I get there and be good to go.
As I’m driving into the parking lot… I noticed that something was looking a bit odd on the street east of the building. Sure enough… a closer inspection revealed that the 25-pair trunk-line that is festooned over the street to the building had been snapped. Apparently an over-sized truck came down the street, snagged it and just ripped it all down and kept going. I took a quick look at the Nagios trend log on the server at home and see that this had happened at 12:59 AM. I get in touch with the ISP who in turn had to get in touch with the upstream provider (Global Capacity). The ISP calls me back to tell me that GC needs to “do more testing” to see what the problem is. Are you KIDDING ME??!?! I asked if it would maybe HELP if I send them PICTURES… so that’s what I did. It’s now 11:30 AM and we’re STILL waiting to hear what Global Capacity is doing. And all this time… my users keep asking “so how long is this going be down?”… “is there anything else we can do?” … “are the phones back up yet?”. :-/