Episode III: The Backup Strikes Back

Somehow I managed to get home, crawl into bed, get 6 hours of sleep, and then get up to immediately drive back to the business site to be there by 9:30 AM. Fortunately my morning schedule was pretty open, so I felt that I could hang around for a couple of hours to make sure everything was going smoothly.

The main goal of the day was to take the data that was backed up to the portable USD drive that the owner had with him and copy it back to the server, maintaining the same folder structure they had before. While I was waiting for him to come in, I walked around and made sure the anti-virus was working correctly, and set up the automatic updates for the virus signatures and the like. Somehow during this I managed to foul up their Internet connection, and had to figure that out for a little while, but after a while we were ready to go.

The owner came in with his USB drive, and I told him to go ahead and start copying the data back to the server. While working on something else, he came out and said he wasn't able to copy the data off of the USB drive.

So I went to his PC and tried to figure out what was going on. The drive was on and working. He had a bunch of files on the drive, but all of his backed up files were in a specific subdirectory. I tried copying the files to the server, and got an error (the exact text I've burned from my memory), but suffice to say it was not the clearest in explaining the problem. So I tried copying from the USB drive to the local machine. Failed. Hmm... now this is not so good.

So maybe it's a permissions issue. I go into the security properties on the USB drive and ah ha! It seems that the permissions have been set for PC\user, which was the settings when the PC was not a member of a domain. Now that the PC is joined to the domain, and he is logging into his domain credentials, obviously the permissions are wrong. So since he's an admin, let's go ahead and take ownership and reset the permissions. So I go to take ownership and set permissions and inheritence, and.... wait. And wait. And wait. Hmm... this is not so quick when setting it on thousands of files. But finally this finishes and I go to copy the file and... still fails.

Hmmm... it's now going on almost 11AM and they still have no data. What the heck is going on? Not a destination permissions issue, as I can't copy anywhere. Dig deeper into the properties of the folder that he's backed up to and what's this? The folder is encrypted? What? AARRRGGGHHHHHHHH! I have no idea how or why this got set, but now I think I'm F*&^#@! What little I do know about EFS is that it is entirely credential based. If the encryption is based on his old PC\user credentials, I have no way to create those back again, as it's based on the original SID. (I may be off here, but what little documentation I could find seemed to back this up.) So now it looks like I have no way to get this data back.

Just to make sure, I try unjoining his PC from the domain, and then logging in back under his old credentials, but I still couldn't copy the files. So now it seems like this data is unrecoverable. Fortunately, I have, as we say in poker, an "out". Since we used new hard drives for the SBS install, I still have the original hard drives (in their static bags, even) sitting on a shelf next to the server. So, while the owner goes out to lunch, I shut down the server, remove the new drives, put the old drives back in, reset the cables back to their original configuration, boot up the server again (to Linux, ugh!), and once the server is back up, I go back to one of the PCs and copy all the server data files to a local folder and make sure I can copy them back somewhere else. Once I have all the folders copied again, I shut down the server, swap the drives and cables again, boot back into SBS, and copy the folders back to the server. Total reconstruction time: 90 minutes. Which given everything else, I didn't think was too bad.

And here's where I get to Epiphany 6.

Epiphany 6: Don't ever trust the end user/customer. If they said they did something, do it again yourself. Better yet, just do it yourself the first time.

If I had actually looked at his backup drive instead of just trusting that it was just a simply copy/move, I probably could have saved myself a lot of hassle.

So now it's close to 1PM, we finally have the data back up on the server, the PCs are joined to the domain, anti-virus is installed and updating. We don't have the server published to the Internet because they have a dynamic IP address, so I purchase No-Ip.com, get that installed on the server so we can keep their DNS records up to date. At the same time, I also need to configure their Cisco 678 DSL Modem to enable port forwarding. Unfortunately, Cisco doesn't really have a web interface, so I'll need to install the Cisco configuration app and learn how to configure port forwarding, but that I'll save for another day.

Long story short, I finally get out of there around 3PM, with the data in place, the server running, and the clients working. And yet there was still so much more to do....