My main image external 1Tb HDD has some issues. Some files don’t read, that makes some folders hang. This drive has also darktable (and some other image software) running.
For the RAW I have backup, that’s not a big deal, just don’t like to lose the darktable db.
CHKDSK fails with an unspecified error occurred (6672732e637878 5df), so I wonder if there are any Linux tools that can help.
The drive is a WD Passport.
TLDR: Replace your harddisk asap
This indicates most likely bad sectors, and if you have lots of them, that indicates a failing disk. In general I believe native windows tools are best for checking ntfs. You can also use a tool that can read smart statistics (on linux: smartmontools, but there’s tools for windows too), however the evidence is already pretty clear.
I don’t know whether chkdsk does it automatically or not, but it should be possible to try and mend the broken segments/sectors or at least mark them as unusable (the latter without bringing back your data, good thing you have backups). Thus the hanging should be gone and you should be able to rescue the good data, such that you only need to recover the broken one. Even if this works flawlessly, I wouldn’t keep using the disk. It’s even a big risk that more corruption will occur/be detected while testing and or recovering data, due to the added disk load.
First thing: Make as complete a disk image as you can before the disk fails completely. You can try recovering files and databases from the disk image later.
I will make a total backup and then maybe do some recovery. darktable still start, so the db should be fine. But this HDD will be phased out.
So something new need to be bought, ideally a 2.5" NAS HDD (NAS + 2.5" is very rare), plus a housing then I am probably already at SSD price level.
I wonder what is more reliable, NAS HDD or SSD?
I had a few HDD die, I never so far had any problem with NAS HDDs, and never with an SDD. However, neither you plug around all the time. I did have some USB flash drives die though.
SSDs are still quite expensive, so I would go for an HDD for “mass storage”. Use the saved money to buy a second one for backup 
. However, as a temporary image storage device for editing SSDs make sense.
Also, HDDs usually die “better”, meaning slower. There can of course always be an abrupt controler failure but often HDD get mechanical problems that show early and give you some time to replace the drive before total failure.
Also, there is this saying: one is none. You should have more than 1 backup. 1TB external drives are $50 USD, which is pretty cheap. Get two of them if you can. I have six, personally.
Read up on apps that do bit by bit recovery with check summing. I have said this on numerous posts that the most important thing is to not use the drive unnecessarily. Keep it unplugged if you have to and only spin it up when you are doing the actual backup.
Yes, something like dd-rescue could possibly recover those bits.
Bad experiences serve as a reminder to make multiple backups using different apps or services. I have botched a few backups of failing drives because a particular app happened to have a version bug or the drive so happened to hiccup badly, causing the copying or imaging to fail.
PS #1 If you are backing up images, bit by bit copying or imaging is important because you could extract partial photos, even partial preview images from raw files. If you know how to access them, it could be a game changer; esp. if you have photos of great sentimental value.
PS #2 If Windows offers to repair any errors, deny that request even though it might sound good; it could cause the data in those sectors to be unrecoverable.
Indeed, external usb HDD do have sudden controller failures, I had one (my first) a couple of weeks ago - the drive identifies in linux but reports no sectors. Fortunately it was only one of several backup copies.
More copies is always better.
I would go here first
https://www.grc.com/intro.htm
an external drive enclosure will help in certain situations for data recovery.
Join the news groups at grc and look everywhere on the net for info on data recovery.
If you are uncertain about doing this yourself and have the money then look here
https://www.rossmanngroup.com/, Louis does data recovery. check his youtube channel
https://www.youtube.com/channel/UCl2mFZoRqjw_ELax4Yisf6w
For now the HDD mainly still works, a few files are damaged. I know, this will get worse in no time. In the meantime NAS type 2.5" drives are hard to find around here. So I probably go for an external SSD - a Sandisk or Samsung 500Gb external SSD is well below $100
I will use it the same way, one drive to stick around on different PCs and keep the darktable db in a folder on the drive - but with an automatic backup of the db folder to the NAS. (dt is still fine, db not damaged, yet)
I don’t need data serious recovery at this point. Some RAW images are bad (hence darktable loads the folder poorly, with errors), but I have backup for the images, and the processed RAWs are in the db - that I like to keep. Plus the folder with the ‘JPG outputs’.
Is there a ddrescue user around? I can’t really work it out yet, does ddr has a analyse only function without doing any repair?
ddrescue is just a copy tool. It will copy your whole partition/drive to an image file or to a second drive. Data rescue is done afterwards with other tools on these copies.
ddrescue differs from dd (that makes exact copies) in doing the “easy” stuff first. So it copies everything first that gives no problems. The difficult parts are done later. In this way it tries to keep the loss as small as poosible because the use of a damaged HDD can lead to total failure at any time.
I have used the following tools a couple of times successfully on Ubuntu 16.04:
ntfsfix - to recover ntfs formatted external drives (esp after a bad/aborted copy session)
photorec - to recover picture files and documents from USB sticks and even SD cards
These could help recover some files from defective drives.
I’ve seen a lot of great recommendations for technical things on this thread.
Having dealt with many failing spinning disks over the years, I have to say:
Also, you should do this before, during, or after — just do it ASAP:
If your backup is the only backup you have, then you don’t actually have a backup anymore. I’ve had disks fail in the middle of recovering from backups.
(In my case, it was on a 2-disk failover on a RAID. The second disk died in the first rebuild. I was nervious for a few days as the RAID rebuilt the second time with 2 new disks.)
BTW: I’m guessing that even doing standard copying from NTFS in Linux will probably go better than doing so in Windows. On Linux, most of the time, it’s a FUSE mount, and Linux usually will throw IO errors and keep skipping until it gets to the next usable chunk of data on the disk.
I haven’t used it, but https://www.system-rescue-cd.org/ looks like a great all-in-one Live USB stick distro with lots of docs. Last release was this month too, which is a good sign.
Oh, thanks for giving me another tidbit of anecdotal evidence on the fallacy of a two-disk redundancy scheme, especially where the two disks are the same make/model and procurement lot. Put 'em in a mirror RAID, and they’ll eventually fail one right after another, the second one probably before you can restore the first one… just like your automobile headlights.
Yep! I even bought from different batches to try to prevent this from happening. And, before this event happened, I was second-guessing my original choice of using 2-disk failover.
But, that said, I think next time around, I’d rather go for 1-disk failover and yet-another backup instead. But it’ll probably be a long time from now anyway. This RAID has been running for several years.
(With all the disks replaced at one point or another. I haven’t had the best luck with hard drives — inside this RAID and outside too. Unless you define luck as lack of data loss, and then I’ve had great luck… but that’s probably more due to backing up everything important, and in multiple places when possible.)
You want another one?
After getting my first NAS (Synology DS212j) I was convinced that NAS HDDs are just a marketing trick to suck extra $$$ from poor us, so I bought two common desktop HDDs.
I don’t recall how long they lasted, quite a while, maybe 2 years. Then one went pretty bad and the 2nd started to get errors.
I saved all data. The problem here is not getting the data - but finding a place to put it. I did learn and bought NAS HDD after that, and they still run flawless.
Hence, if I go for external HDD again I would get a DIY NAS HDD.
However, IMHO that usually slow death of an HDD is a huge advantage. A USB memory stick just goes bad in a microsecond, and I guess that a portable SSD might work similar.
Quick answer: fire a live Linux distribution and use a command called ddrescue to make a physical image of your HDD. I will give you more instructions later if you are interested .Do not use the device meanwhile…
@Carmelo_DrRaw: What happens when a drive fails during making a disk image? (It would be an incomplete image and probably won’t have the most important data. In some cases, it might not even mount properly.)
That’s exactly why I suggested grabbing any irreplaceable files while it’s still (probably) possible first. Taking an image of the drive afterward is great, of course, if the drive is still responding.