Of back-ups and screw-ups… (and how to recover)

My dear sister had asked me to please reinstall everything on her laptop as it was heavily polluted with all kinds of residual software from the era my dad used to use it. I am always more than happy to help out my little sister with that kind of stuff.

She is gone with my parents to South Africa (chasing Lions n’ stuff) so this was the perfect opportunity for me to make things right for her. I picked her laptop up the evening before the left and the CDs to reinstall it all.

Little did I suspect the calamity that was on my path.

Like any other sane person would do I first backed up all important data. Since I have a little file server that seemed the logical place to put the files. After reinstalling Windows (after fighting with the built-in CD-ROM drive) and other software I wanted to put the data back. Except it was nowhere to be found.

You must understand that I on that very moment got a very sick feeling in my stomach and started sweating profusely. I had no clue to as what the hell had happened. I only knew one thing: the directory structure was still there, but all files were gone.

After a little while it began to dawn on me that in a distant past I had a job running that would clean out old backups and that it might be running still. I checked and, yes, it was still running every night. By this time I was feeling physically ill.
I assume at this point that, since the script was set to run and check for modified time and all files I had backed up on that
partition, the script automatically started to wipe everything out, since the modified date of the files exceeded the retention I had set up at one point.

I had fucked up… Big time.

After swallowing down my nausea I wondered what I could still do to undo the mess I got myself into I started looking into some forensic methods of recovering files. There was to my surprise a lot of freely available tools for recovering data from drives. However one quote made me even more desperate. The quote is from Andreas Dilger, one of the developers of EXT3:

In order to ensure that ext3 can safely resume an unlink after a crash, it actually zeros out the block pointers in the inode, whereas ext2 just marks these blocks as unused in the block bitmaps and marks the inode as “deleted” and leaves the block pointers alone.

Source: http://batleth.sapienti-sat.org/projects/FAQs/ext3-faq.html

That’s already 2-0 for the gremlins. My backup partition was formatted with a EXT3 filesystem. No blocks information left to the inodes I could uncover using fls. icat could not extract any information. I tried to get something using a tool called photorec but the results are poor. Data can be salvaged, but at this point only as chunks of data to be analysed, not as the original files itself. A bloody mess.

Restoring the backup at this point seems like a lost cause. This leaves me the option of the laptop’s harddrive.

fls and icat also seem to work on fat/ntfs type file systems and according to what I have read should be able to work better than trying to recover files on ext3. All I need to do is get a bootable Linux Live CD and install The Sleuth Kit software to start recovering.

Unfortunately, my sister’s laptop is being a bitch and the CD-ROM drive will not work properly. There’s no booting from Knoppix CD or any CD for that matter (reinstalling Windows was a bitch to begin with). So another score for the gremlins.

On a more positive note, Photorec also offers an executable for working under Windows and even with a new file system in place it is possible I could salvage some of the data. Photorec seems to do a good job extracting images so far. Unfortunately I cannot say the same about other file types (like word documents) but the scan is still running.

I can only hope this final try will yield better results. I am hopeful of recovering photo’s from my sister but I have no illusions about data being lost forever.

I am still feeling terrible, since obviously I am directly responsible for this mess and even have to still tell my sister. No matter how many soothing words I hear from my sweet wife there is no relief, only acceptance and remorse. This is one of these events in your life, that will only get less of a burden over time.

If you read this dear sister, I hope you will not be mad at me for too long. 😉 😀 🙂