What transcoding settings should I use?

[Donát Frink]

So I started using Jellyfin and I really like it but I don't know what transcoding settings I should use because my server is not only weak but when I try to watch anything in HEVC format it says "fatal player error". I have close to zero experience in setting up transcoding settings so I need a bit of help.

My specs are:
OS: Debian 12
CPU: Intel I3-3220
GPU: Intel HD Graphics
RAM: 16 GB

[Host-in-the-Shell]

With a series that old (I believe it's Intel Ivy Bridge which is 3rd gen), the only encoding/decoding available would be H.264. So long as your media collection is in that format or can be converted, you should be able to have some transcoding capabilities. That said, I don't own any intel hardware that is older than 2016ish so I can't really speak for its transcoding performance or potential limitations (if any).

[theguymadmax]

That IGPU is very old and has limited hardware abilities as Host in the Shell pointed out. Under Hardware Acceleration you need to select Video Acceleration (VA-API) and select the decoders as specified on this page. For encoders only h.264 should be enabled.

If you want you can post a screenshot of your transcode settings.

[mildlyjelly]

Just to be clear, since you are new to transcoding (I'm sure you already know at least some of this).
Transcoding is the process of changing a video's encoding from one format to another.

So, if your video is encoded in the HEVC format, for example, and you try to play the video on a device that doesn't understand HEVC (which is annoyingly common), you will get an error.

Jellyfin is capable of detecting that your device doesn't understand HEVC, and can transcode the video into a format it does, such has H.264.
The transcoding can be done with either "software encoding" or with "hardware acceleration". Hardware acceleration uses specialized hardware (typically built into GPU's) to perform the transcoding. Software encoding using the CPU to perform the transcoding (technically this can still be done on a discrete GPU).

Because the hardware acceleration uses specialized hardware, when a new format comes out, you would need new specialized hardware to support it.
When you are missing the specialized hardware, Jellyfin can fall back to software encoding, which doesn't require any special hardware.

This chart here (I'm pretty sure it is the correct one for you IGPU), shows that your GPU supports H.264 hardware decoding, but not encoding.
   
https://www.intel.com/content/www/us/en/...vaapi.html

So, in your case, your transcode settings should be set to VAAPI.
For hardware decoding, enable H.264 and MPEG2 (all other check boxes should be disabled).
All hardware encoding options should be disabled.

[Efficient_Good_5784]

Here's another way of checking which HWA options are supported by your Intel CPU.

1. Google your Intel CPU and get its specs from Intel's website: https://www.intel.com/content/www/us/en/...tions.html
   
2. In the specs, check what its codename is. In this case: "Code Name: Ivy Bridge"
   
3. Use the Wikipedia table to reference which options the iGPU in your CPU supports for HWA: https://en.wikipedia.org/wiki/Intel_Quic...d_encoding
   

Going off of that....

Enable these for decode:

1. H264
   
2. MPEG2
   
3. VC1
   

Disable these for decode:

1. HEVC
   
2. VP8
   
3. VP9
   
4. AV1
   
5. HEVC 10bit
   
6. VP9 10bit
   

As for the hardware encoding options, make sure to disable both options (HEVC & AV1) since the iGPU doesn't support them.

[theguymadmax]

I think mildyjelly is correct from this link.

Linux VA-API supports nearly all Intel GPUs.  - I overlooked the word nearly and just read all.

Linux QSV supported platforms are limited to: - Ivy Bridge is not supported.

[mildlyjelly]

Based on the link Efficient_Good_5784 provided: https://www.intel.com/content/www/us/en/...tions.html
I think your IGPU does support Intel Graphics HD 2500, which would mean I am off a row in my chart. So, the GPU probably does support H.264 hardware encoding.
So, I think your configuration should look like this (minus the VA-API Device setting. Yours is probably different):
   

Enable hardware encoding, see if it works, if you still have problems, disable it and see if that fixes it.

[Donát Frink]

Thank you guys! The suggestions have worked wonders, now I can watch movies without this poor pc dying😂.