Viewing a single comment thread. View all comments

KRed75 t1_ix1fck4 wrote

I once pulled up a tile in our data center and there was 2" of water on the floor. I don't know how long it was like that but it didn't cause a single outage. I located the broken pipe under the raised floor, installed a patch and grabbed the sump pump. Ran a hose outside and started pumping water.

We kept the patches, sump pumps and hoses specifically for a situation like this.

Nobody ever said a word about it to me or anyone else which is surprising because we had thousands of servers for hundreds of customers all over the world in that data center. Now we only have dozens of servers with shitloads of RAM and CPU cores running virtualization software to handle about 10 times the number of customers. I haven't been in the data center in 15 years. We used to have hundreds of people in that building. Now it's only about 20. We walk them through handling hardware installations, removals and repairs and we do everything else remotely.

5