trouble streaming hevc videos

[lanwin]

Hi, I have trouble streaming hevc videos on my homeserver. I already searched the web but did not found anything useful that matches my case.

H264 videos are play just fine (even jumping to positions works) but my hevc videos are producing the following errors:

Code:

[hevc @ 0x5ed64eb82b40] PPS id out of range: 0
[hevc @ 0x5ed64eb82b40] Error parsing NAL unit #0.
[vist#0:0/hevc @ 0x5ed64ea5e5c0] [dec:hevc @ 0x5ed64e8b4fc0] Error submitting packet to decoder: Invalid data found when processing input

Some videos play the first few seconds and then stopping. All videos are encoded via HandBreak and working fine in VLC on Windows.

My first guess was HW transcoding, but even when I disable hardware decoding/encoding via admin UI the problem persists.

My setup:
* Jellyfin 10.10.3 
* docker container 
* host OS ubuntu 24
* Using integrated graphics of my Ryzen 4750G CPU

[TheDreadPirate]

VLC will move heaven and earth to play corrupted videos. The fact that it works in VLC is not a good benchmark.

How do you have transcoding setup now? The ffmpeg probe in your log says the video is SDR, but the error messages are logs I usually see with HDR videos.

Was this originally an HDR video prior to encoding with Handbrake?

[lanwin]

Thank you for your answer!

Well I encoded some of my vidoes with "H.265 10-bit (NVEnc)" cause I thought that Handbreak does the right thing even when the source is SDR.

I am also not sure where in Handbreak I can see if the source format is HDR or SDR. Most of my source material are Blueray's.

Is there a way to fix this without encode the videos again?

[TheDreadPirate]

It depends on what kind of HDR the original video was. Assuming the original video was HDR (I think it was).

Handbrake's documentation says that only HDR10 is supported with H.265 10-bit NVENC. If the original video is HDR10+ or Dolby vision, you'd have to use x265.

https://handbrake.fr/docs/en/latest/technical/hdr.html

And you would need to re-encode.

Can you share the media info for the original video?

[lanwin]

Thank you for your answer!

Well I encoded some of my vidoes with "H.265 10-bit (NVEnc)" cause I thought that Handbreak does the right thing even when the source is SDR.

I am also not sure where in Handbreak I can see if the source format is HDR or SDR. Most of my source material are Blueray's.

Is there a way to fix this without encode the videos again?

It seems they are not HDR since I use mostly normal Bluerays and not UHD's. If I understand it correctly, it seems that HDR and HDR+ are only supported for UHD's.

I also found the place where I can see if its SDR. It says SRD (8-bit 4:2:0, 1-1-1).

So I guess the problem is more that I encoded SDR content with and HDR encoder.

The only question remains it if I can remux that without reenocde it to fix that?

[TheDreadPirate]

H265 10-bit NVENC is not an HDR encoder. It just supports it some kinds of HDR content.

I used Handbrake for a long time before moving to ffmpeg for pre-transcoding. Also with "H.265 10-bit NVENC" as my encoder, even for SDR content. Without knowing all the options you selected Handbrake, I can't say what the root cause is.

AFAIK, Handbrake cannot remux without re-encoding. But ffmpeg can. You can use ffmpeg running in your container.

Code:

sudo docker exec -it jellyfin bash
/usr/lib/jellyfin-ffmpeg/ffmpeg -fflags +genpts+igndts -i "/path/to/problem/video.mkv" -map 0 -codec copy -max_muxing_queue_size 2048 -max_interleave_delta 0 -avoid_negative_ts disabled newfile.mkv