JF Server storage config advice?

[Lexx]

ISP bandwidth limit: 1000mbit up/down

PC Hardware available:
-Core 5th gen Intel CPU, 16GB DDR3, 2x 16TB 5400RPM HDD, WD Mycloud Ex4 4-bay NAS.  1x1TB NVME SSD.  Radeon 380 gpu.

My own competency:
-mostly Windows.   Some command-line stuff.  Some history as a Windows sysadmin but we're talking Server 2003 era.  Very VERY little linux.   Willing to play with Linux but a bit nervous about it.  

Goals:
-maximize reliability and data loss prevention.  Ideally I'd like the storage array to survive the failure of ANY component.  OS, motherboard, drive controller, individual drive, etc.
-Concurrent streams: 50+

Any ideas?

[JustAnotherDude]

I would use Linux, distribution is not terribly important. mdadm to setup your your two HDDs in a mirrored raid array, giving you a single 16tb storage that will persist if one drive fails, and no extra raid controller needed.

Your upload speed and drive read speeds can likely handle 50+ streams at a low bitrate, but I am skeptical that your cpu can handle it, and I don't think the old amd gpu will handle transcodes particularly well.

[TheDreadPirate]

For Jellyfin's app data, just making regular backups is the preferred method to survive hardware failures.

For your media, you'd have to use, preferably, ZFS RAIDZ1 or 2. ZFS RAIDZ1/2 does not rely on a hardware controller and can be imported into a new system if you had to migrate the hard drives or rebuild the OS, or move the drives to a new NAS enclosure.

Not sure if your NAS can do that or not. I know for certain that Synology supports ZFS raid levels with parity. QNAP, Asustor, and UGreen almost certainly support ZFS raid as well.

[34626]

I would not use RAID, i have mutiple dirves but 2 servers. One in front and another as backup.

If server 1 goes down, server 2 is identical to server 1 and can take over while i fix server 2.

Currently my server is just a intel NUC N5105 with 4 external drives and one SSD in it..

But i do not have 50 users, so my backup is just a raspberry pi 4 connected to some other drives then server 1.. I havent had any down time due to hardware failures yet.

[TheDreadPirate]

I would not use RAID, i have mutiple dirves but 2 servers. One in front and another as backup.

If server 1 goes down, server 2 is identical to server 1 and can take over while i fix server 2.

Currently my server is just a intel NUC N5105 with 4 external drives and one SSD in it..

But i do not have 50 users, so my backup is just a raspberry pi 4 connected to some other drives then server 1.. I havent had any down time due to hardware failures yet.

When you don't have enough space, money, etc., to do a full backup of your media, your remaining option is RAID.

Disclaimer: RAID is not a backup!  But money doesn't grow on trees so backup properly smaller data sets (Jellyfin/other container app data) and hope that RAID is good enough to survive hard drive failures.

Since most of us are only running servers for immediate family and not commercial setups (RIGHT!?) the need for a spare server to eliminate downtime is not worth the expense.

[34626]

I would not use RAID, i have mutiple dirves but 2 servers. One in front and another as backup.

If server 1 goes down, server 2 is identical to server 1 and can take over while i fix server 2.

Currently my server is just a intel NUC N5105 with 4 external drives and one SSD in it..

But i do not have 50 users, so my backup is just a raspberry pi 4 connected to some other drives then server 1.. I havent had any down time due to hardware failures yet.

When you don't have enough space, money, etc., to do a full backup of your media, your remaining option is RAID.

Disclaimer: RAID is not a backup!  But money doesn't grow on trees so backup properly smaller data sets (Jellyfin/other container app data) and hope that RAID is good enough to survive hard drive failures.

Since most of us are only running servers for immediate family and not commercial setups (RIGHT!?) the need for a spare server to eliminate downtime is not worth the expense.

Did you even read the post? There is 50+ clients and nothing about a limited budget, so why do you nack on my suggestion that meets Lexx's goals?
If money is limited, then you save up to buy drives when they are cheap or there is a great offer... I dont know how you do, but it aint free to be self-hosting, but it does give you freedom and possibilities.. Also, Jellyfin is just a service, i dont know about you, but i do have other services runnning on my server..

[Swindiff]

If using windows look into Stablebit Drivepool.  Its what I use to create my main media server array (10x8TB drives)  I don't use it for redundancy, I regularly run backups to 2 x other servers also running Stablebit Drivepool Arrays.  The advantage with Stablebit over Windows storage space is that if a drive goes down, you only lose the data on that drive rather than trashing your whole array (If you don't have any redundancy)

You can however set it up to make multiple copies of files to give you some redundancy if just using the one server.

[TheDreadPirate]

@34626 The attitude is not appreciated. The fact that they mention "available hardware", and the available hardware is fairly old, tells ME that they have a limited budget. They also don't mention anything about buying new hardware which, again, tells me they are trying to make due with what they have and how to optimally configure their existing hardware.

Having said that, 50+ users would require that you disable transcoding as transcoding at that scale is not viable for that hardware. Even modern and more capable hardware usually caps out at ~20 concurrent transcodes (for video).

1080 H264 8-bit video
Stereo AAC
SRT subtitles

The above codecs would ensure broad client compatibility without the need to transcode.

[Lexx]

Concerning transcoding, I'm basically allowing for now and going to be setting up event tracking and logging. If it becomes a problem, I'm going to be pruning any content that requires video transcoding. Real-time transcoding is power inefficient and gawdawful quality. At the moment I'm playing with AV1 with an eye for the future. (so sad that the AV1-SVT-PSY fork is essentially done, but so grateful for its work!). Any spare CPU cycles are going to be powering transcodes at placebo level detail retention goals. Like preset 1, RF20 stuff.

And, honestly, not planning on having 50+ simultaneous streams, but I'm impatient and annoyed by bottlenecks. If my internet connection can handle...say 80% of its rated throughput, that's 800mbit/s. Take away 200mbit/s for random other uses. That should leave AT LEAST 600mbit/s in available bandwidth. That's...85 simultaneous streams at 7mbit/s, or 17 simultaneous streams at 35mbit/s. Not that I'll ever need that, yeesh, but it SHOULD BE ABLE TO DO SO.

I've also spent an stupid amount of time trying to turn off power management for my existing WD external drive, because it spins down when not actively in use, and opening up jellyfin or trying to read from it causes it to spin up which takes a good 5-10 seconds. IT SHOULD BE INSTANTANEOUS.

the biggest bottleneck in my available hardware is HD seek time, and I'm seriously considering throwing a 1TB NVME drive into the computer with 700gb of it allocated to PrimoCache (software that intelligently turns SSDs into, essentially, huge page files that intelligently cache crap-tons of frequently accessed files and folders).

Anyway. That was a surprisingly long rant. How I think I'm going to proceed for now, after some thought:

OS: Win10/11, existing license. NVME drive running OS partition and second partition as HDD cache. Media library will EITHER be RAID1 using the NAS and the two dissimilar 16TB drives, OR using Windows storage spaces RAID1 config.
Preference will be to whichever configuration allows HDSentinel or CrystaldiskInfo to realtime monitor drive health and send alerts when things start to deteriorate. Either way we're going to go with ReFS for now. My understanding of RAID1 is such that if I lost the NAS or either of the drives, I still have an intact copy (or two!) of the data that should be plug & play into any storage controller. I'll be pulling nightly OS image backups and more frequent JF data backups automatically.

Going the Caddy/dynamicDNS route and setting windows firewall rules to essentially block EVERYTHING save admin remote desktop access and JF/Caddy.

This will be the interim server while I experiment with the linux setup.
Linux setup will be...I'm not sure, UNRAID or trueNAS scale...not really sure. But the idea is there will be multiple isolated docker containers. One for storage controller, like truenas or unraid, etc. one for reverse proxy (Caddy), one running JF, one running JF config test system, and one running transcoding jobs.

More than anything I'm looking to set it and forget it, and any catastrophic failures will be, at worst, 30-60 minutes of my time to restore.