2024-10-25, 03:10 PM
(This post was last modified: 2024-10-25, 03:31 PM by Host-in-the-Shell. Edited 3 times in total.)
So over the years I've come to purchase, sell, repair (as little as you can, in reality), clean up and inspect many drives. I've been using various kinds of them since 2005-ish, and currently own for personal use about 20 of them. I also sell PC parts and deal with drive batches constantly, new, used and so called "refurbished". More on that later. My record of a drive running for the longest time was a Western Digital 500gb one that ran from 2011 to around 2022 when I de-commissioned it. Maybe it could have run for longer, but I often cycle drives so it was about time. Again, more on that later.
I'm not an expert by any means, nor am I going to go in depth with the types of drive you should consider and what benefits you can have when getting one or the other. There are already a million online guides and videos far better than I could ever break it down anyway, and the point of this little guide here is to focus on the practice of buying the drives themselves. In short, I'll assume you want your drives for the purposes of storing your Jellyfin media, and under that assumption you want NAS grade or comparable drives, and likely you'd want to avoid Seagate if you can. In my experience, for long term storage, these tends to fail often.
If you don't want to read through this extensive post, you can find a conclusion/TL;DR at the end.
For the purposes of this tip list, I'll categorize the media that you're going to store in two possible ways: critical and non critical.
So now it's time to dispel some myths:
Now, I'll breakdown drive manufacturers according to my experience over the years. Again, this is just personal experience so take it with a grain of salt. If I leave any brand out is because I haven't used their drives.
Now, let's get into new vs used/"refurbished".
So you probably heard before that you should NEVER get a drive second hand. And while this is great practice for production and otherwise any critical data, media storage can often be non critical. You either don't care about losing your collection, or can easily copy it over from elsewhere. In that case, buying extremely expensive new drives can quickly burn a hole through your wallet. There's also the fact that drives will fail on you regardless of the condition they are listed on. All drives fail, and when and where can depend on many factors, and break expectations against or in favor of their operation. The real benefit of new drives are the warranties, which usually run longer and include data recovery services as well as other type of support. Second hand drives can have these too but often on more limited capacity.
Ultimately, buying new or not will depend on your use case. For this guide, I've assumed your media hosted on JF is non critical, so second hand drives will bring the added benefit of not costing you as much, and not having you pull your hair out if there happens to be data loss.
Your drive is not always dead, even if it appears to be.
At the first sign of trouble if you still can, you should RMA or get refunds; but if you happen to be outside any return or support window, your drive could still be rescued. Many times a failed drive is not really faulty, just needs a new controller. If you can test for that and replace it, this could be very handy. Just beware if you decide to find a technician to do this for you, since there are many shady repair centers that will charge you ridiculous prices for any routine and simple service.
Monitor the health of your drives periodically irrespective of if new or not.
You can take preventive measures sometimes months in advance if you keep an eye out for your drives. Thoroughly testing them once you get them is great, but you should still track them afterwards and pay attention to any log events that are pertinent to them.
Update your firmware.
Sometimes bad sectors or errors are reported due to firmware bugs or errors. You should keep the firmware updated for security and stability. It can be a pain under Linux, but there are usually workarounds.
Beware of bit rot.
Your files will degrade over time; running a checksum on them after several years will let you know of this fact. Other than aggressive cold storage solutions that can run you in the thousands of dollars such as magnetic tape storage, there's little we can do to mitigate this. However, one great practice is to cycle your drives.
Cycling drives avoids bit rot, allows duplication and mitigates data loss.
If your data is critical, then your commercial grade drives should be cycled after the 3 year mark, while nas and enterprise grade should be done so around the 5 year mark. Cycling a drive means replacing it with a new one of the same size, and keeping the older around as additional backup just in case. Notice I said "additional" as you should always have backup already in place if you can't afford to lose it. This practice has bailed me out so many times, it's crazy
Actually, there's disc rot too.
Physical media can be quite handy for backups, such as discs (CDs, DVDs, BluRays, etc.) but keep in mind that they also degrade over time, particularly if they are not pressed. It can also rack up a hefty bill over time.
Spin up cycles can be far more damaging than 24/7 run time.
If you spin down and back up your disks frequently, you will experience failure at a significantly earlier rate. One problem many face unknowingly comes from disk sleep, which forces these cycles on your drives. While disabling sleep mode will increase power consumption, it will lead to longer life spans.
Conclusion, or TL;DR
Buying disks, and in which state, should be done according to how critical the data you will write to them is. Use common sense: check and double check warranties, return periods, support windows and always read the fine print, if there is any. Compare and contrast prices and condition in light of your use case. For me, HGST and WD have been the best brands, but you should do your own research too. Test each disk you get with a physical inspection as well as smart test; always RMA at the sign of trouble if you can, as it is your right. Keep your firmware updated; if your drive dies, check if replacing the controller can save it, provided you are outside any return or support window. Beware of bit rot and disc rot, and if your data is critical, always implement a 3-2-1 backup strategy as well as a drive cycling program: 3 years for commercial grade drives, 5 for NAS/Enterprise. Make sure that you reduce spin up cycles and disable sleep on your disks if you want to ensure a reduced failure rate.
This was mostly all off the top of my head and intended for casual buyers/new hosters of JF; probably lots of things can be added here, so feel free to chime in with your ideas and advice.
I'm not an expert by any means, nor am I going to go in depth with the types of drive you should consider and what benefits you can have when getting one or the other. There are already a million online guides and videos far better than I could ever break it down anyway, and the point of this little guide here is to focus on the practice of buying the drives themselves. In short, I'll assume you want your drives for the purposes of storing your Jellyfin media, and under that assumption you want NAS grade or comparable drives, and likely you'd want to avoid Seagate if you can. In my experience, for long term storage, these tends to fail often.
If you don't want to read through this extensive post, you can find a conclusion/TL;DR at the end.
For the purposes of this tip list, I'll categorize the media that you're going to store in two possible ways: critical and non critical.
- Critical: Think family photos and videos, rare media that you don't have the source of and cannot afford to lose, etc.
- Non Critical: Anything you can copy over from source, redownload or have backed up properly is considered non critical. It's also any media you don't care to lose.
So now it's time to dispel some myths:
- You can "refurbish" a drive. This is mostly false; the most you can do is clean it up, and replace the controller. There's simply no way to properly factory refurbish a drive without essentially manufacturing it from scratch. In the end, wear is wear, and you can't undo it.
- All "refurbished"/used drives will be DOA or last only a little bit until they go bad. This is also a myth. In reality, the drive lottery in my experience happens regardless of the state of the drive. I've had new drives that are DOA or go bad in weeks, and used ones that lasted many years.
- Only new drives have warranty. This is also untrue; that said, warranty terms for used ones vary depending on the site you use to purchase them and who the sellers are. Always double check your warranty for any drive you will buy regardless of condition on the listing.
- All drives are sold in their stated condition. The more drives you purchase, you will find out real fast that this is not always the case. Do not brainlessly buy "new" drives and call it a day without testing them first; any drive you acquire should be monitored and rigorously studied using the smart data. A physical inspection is also important, particularly to assess age and potential damage during transportation.
- Raid is backup. It is not; not even Raid 1 or any other overtly redundant mode. The only real backup is going to be a 3-2-1 backup strategy. What happens if all your drives fail at once? Sound unlikely? Think flood or other natural disasters. Still too far out there? How about somebody breaking in your home and stealing all your drives? And many other scenarios, really.
- I need a 3-2-1 backup strategy for all my files. You do not; only critical files should be backed up that way. There's no point in employing that scheme for say, Linux ISOs.
- Only irrecoverable data loss is devastating. Sometimes losing data can be crushing even if you have it backed up in some form. An example would be if you ripped from source thousands of media files that you meticulously encoded, and then you lose them. Sure, you still have the sources, but losing all that work can be equally tough, so measures to preserve that effort should be taken.
- Bit rot doesn't exist. Although its effects are often exaggerated, it is very much real and will affect your files over time.
Now, I'll breakdown drive manufacturers according to my experience over the years. Again, this is just personal experience so take it with a grain of salt. If I leave any brand out is because I haven't used their drives.
- Seagate: They make good disks, but their failure rate is the highest, in my experience; most of my dead drives come from them. Their IronWolf line, for instance: I already have a fresh drive developing bad sectors after a few days. Just a little too much of a gamble in my opinion.
- Toshiba: Great drives but they sometimes tend to come DOA, even new ones. Provided you don't mind returning a couple here and there, once you get good ones they last long enough.
- Samsung: I've mostly used their SSDs and nVMEs, and these are good for hosting your operating system and the Jellyfin service. That said, watch out for some of them not being as advertised: Samsung has the tendency to ship other drives under the hood to you when they are short on parts, so test and RMA as necessary.
- HGST Pre-WD acquisition: Excellent drives, but some of these are becoming pretty old by now. It's true they are tanks, but I'd move on to newer solutions if possible.
- HGST Post-WD acquisition: Great drives, but can get flaky sometimes around the 4-5 year mark.
- Western Digital: My personal favorites and the ones that have lasted the longest on me. That said, I hate that WD has been jacking up prices and not providing A grade customer service in recent years.
Now, let's get into new vs used/"refurbished".
So you probably heard before that you should NEVER get a drive second hand. And while this is great practice for production and otherwise any critical data, media storage can often be non critical. You either don't care about losing your collection, or can easily copy it over from elsewhere. In that case, buying extremely expensive new drives can quickly burn a hole through your wallet. There's also the fact that drives will fail on you regardless of the condition they are listed on. All drives fail, and when and where can depend on many factors, and break expectations against or in favor of their operation. The real benefit of new drives are the warranties, which usually run longer and include data recovery services as well as other type of support. Second hand drives can have these too but often on more limited capacity.
Ultimately, buying new or not will depend on your use case. For this guide, I've assumed your media hosted on JF is non critical, so second hand drives will bring the added benefit of not costing you as much, and not having you pull your hair out if there happens to be data loss.
Your drive is not always dead, even if it appears to be.
At the first sign of trouble if you still can, you should RMA or get refunds; but if you happen to be outside any return or support window, your drive could still be rescued. Many times a failed drive is not really faulty, just needs a new controller. If you can test for that and replace it, this could be very handy. Just beware if you decide to find a technician to do this for you, since there are many shady repair centers that will charge you ridiculous prices for any routine and simple service.
Monitor the health of your drives periodically irrespective of if new or not.
You can take preventive measures sometimes months in advance if you keep an eye out for your drives. Thoroughly testing them once you get them is great, but you should still track them afterwards and pay attention to any log events that are pertinent to them.
Update your firmware.
Sometimes bad sectors or errors are reported due to firmware bugs or errors. You should keep the firmware updated for security and stability. It can be a pain under Linux, but there are usually workarounds.
Beware of bit rot.
Your files will degrade over time; running a checksum on them after several years will let you know of this fact. Other than aggressive cold storage solutions that can run you in the thousands of dollars such as magnetic tape storage, there's little we can do to mitigate this. However, one great practice is to cycle your drives.
Cycling drives avoids bit rot, allows duplication and mitigates data loss.
If your data is critical, then your commercial grade drives should be cycled after the 3 year mark, while nas and enterprise grade should be done so around the 5 year mark. Cycling a drive means replacing it with a new one of the same size, and keeping the older around as additional backup just in case. Notice I said "additional" as you should always have backup already in place if you can't afford to lose it. This practice has bailed me out so many times, it's crazy
Actually, there's disc rot too.
Physical media can be quite handy for backups, such as discs (CDs, DVDs, BluRays, etc.) but keep in mind that they also degrade over time, particularly if they are not pressed. It can also rack up a hefty bill over time.
Spin up cycles can be far more damaging than 24/7 run time.
If you spin down and back up your disks frequently, you will experience failure at a significantly earlier rate. One problem many face unknowingly comes from disk sleep, which forces these cycles on your drives. While disabling sleep mode will increase power consumption, it will lead to longer life spans.
Conclusion, or TL;DR
Buying disks, and in which state, should be done according to how critical the data you will write to them is. Use common sense: check and double check warranties, return periods, support windows and always read the fine print, if there is any. Compare and contrast prices and condition in light of your use case. For me, HGST and WD have been the best brands, but you should do your own research too. Test each disk you get with a physical inspection as well as smart test; always RMA at the sign of trouble if you can, as it is your right. Keep your firmware updated; if your drive dies, check if replacing the controller can save it, provided you are outside any return or support window. Beware of bit rot and disc rot, and if your data is critical, always implement a 3-2-1 backup strategy as well as a drive cycling program: 3 years for commercial grade drives, 5 for NAS/Enterprise. Make sure that you reduce spin up cycles and disable sleep on your disks if you want to ensure a reduced failure rate.
This was mostly all off the top of my head and intended for casual buyers/new hosters of JF; probably lots of things can be added here, so feel free to chime in with your ideas and advice.
Server specs => OS: Debian 12 | GPU: Arc A380 | CPU: Ryzen 5 5600X | 64GB RAM | 56TB
