WebOS Struggles w/ DS4K+HDR Content

[bitmap]

I'll provide a log in just a few but writing this out will take me a minute so I wanted to get that done.

The media in question is downscaled (4K --> 1080p), encoded using av1_qsv (g:q 22) with jellyfin-ffmpeg7, HDR metadata retained, DV removed using dovi_tool (if applicable), audio converted to 256k Opus. The overall bitrate for most of these is 2000-6000 kbps. The media is direct playing, no PGS/ASS/VOB subs enabled. This does not happen with every piece of media given the aforementioned treatment, but enough of them to be problematic.

The audio works just fine throughout viewing, but the video stream appears to get "stuck". Quick FF using right direction on remote will generally bring the video in sync with the audio at that timestamp, but it always sticks again (i.e., freeze-frame). If you leave the session alone, eventually the video "fast forwards" itself to catch up with the audio, only to freeze again momentarily. We're talking 30s to 2m of freeze frame, all frames displayed and screaming at a couple hundred frames per second to catch up with the audio. To be clear: this is 100% not a network issue. I can stream 20 mbps 4K content, even remuxes at 60+ mbps without issue. This seems to be a problem with the media, the encoding process, or how the WebOS client is handling the media. It's extremely odd to me...

I have tested with my Android phone, friends have utilized several different iPhone models, and even a Roku 4K stick -- on a TV which requires tonemapping -- and the media plays without issue. My phone is a Pixel 6, which supports HDR so there is no tonemapping or transcoding. The reason for treating media in this fashion is that the majority of clients support HDR, phones that support HDR are commonly-used hardware, and the bitrate (read: file size) to visual clarity is absolutely unrivaled.

So my question here relates to why this might be happening -- not enough keyframes? Chroma issues from downscaling? Has anybody seen this behavior before with any kind of content and found a fix?

ffmpeg command for anybody interested:

Code:

MEDIA="MEDIA" && INPUT="INPUT" && \
cd "/mnt/media/luxe/downloads/complete/re-encode" && \
mkdir -p "/mnt/media/luxe/import/${MEDIA}" && \
# dv_rm is an alias for --> jffmpeg -init_hw_device qsv=qs:/dev/dri/renderD129 -i "$1" -c:v copy -bsf:v hevc_mp4toannexb -f hevc - | dovi_tool remove -;
dv_rm "${INPUT}" && \
jffmpeg -init_hw_device qsv=qs:/dev/dri/renderD129 -hwaccel qsv -hwaccel_output_format qsv -qsv_device /dev/dri/renderD129 -filter_hw_device qs \
  -analyzeduration 200M -probesize 1G \
  -i "BL.hevc" -i "${INPUT}" \
  -map 0 -map 1:a:0 -map 1:s:0 -map 1:s:1 \
  -c copy -c:v:0 "av1_qsv" -preset "veryslow" -low_delay_brc 1 -extbrc 1 -look_ahead_depth 100 \
  -global_quality:v:0 22 \
  -filter:v:0 "hwupload=extra_hw_frames=64,vpp_qsv=scale_mode=2:w=1920:h=-1:denoise=6" \
  -filter:s "hwupload=extra_hw_frames=64,vpp_qsv=scale_mode=2:w=1920:h=-1:denoise=6" \
  -c:a libopus -b:a 256k -ac 6 -filter:a aformat=channel_layouts="7.1|5.1|stereo" \
  -metadata title="${MEDIA}" -metadata:s:v:0 title='AV1 Encoding by bitmap' \
  -metadata:s:a:0 title='5.1 Surround' -metadata:s:a:0 language='eng' -disposition:a:0 default \
  -metadata:s:s:0 title='English' -metadata:s:s:0 language='eng' -disposition:s:0 0 \
  -metadata:s:s:1 title='English (SDH)' -metadata:s:s:1 language='eng' -disposition:s:1 0 \
  "/mnt/media/luxe/import/${MEDIA}/${MEDIA} [Bluray-1080p AV1 HDR OPUS 5.1][EN][EN]-RLSGRP.mkv" && rm BL*.hevc

[TheDreadPirate]

I had a ton of issues getting jf-ffmpeg7 to work with the dovi_tool extracted HDR10 HEVC files. Reverting to jf-ffmpeg6 "solved" the problem. I spoke with the devs a bunch to try to work it out, but what ended up happening was Gnattu wrote and integrated a function directly into Jellyfin-ffmpeg to remove Dolby Vision.

https://github.com/jellyfin/jellyfin-ffm...g/v7.0.2-4
https://github.com/jellyfin/jellyfin-ffmpeg/pull/480

As long as you are on version 7.0.2-4 and newer, you don't have to bother with dovi_tool. I updated my DV to HDR10 guide to account for this.

https://forum.jellyfin.org/t-converting-...n-to-hdr10

Bonus feature, if you encode to AV1 with 7.0.2-4+ it will automatically strip the Dolby Vision EL and RPU for profiles 7 and 8. If you did want to just remove DV, the command specified in my guide would be the way to do it with the bitstream filter.

Code:

-bsf:v hevc_metadata=remove_dovi=1

But with your ffmpeg command, you shouldn't need to bother with BL.HEVC. Just feed in the original video and adjust your mapping.

[bitmap]

This is literally all of the output of the log while viewing the first five minutes of one media file:

Code:

[2024-11-08 11:36:21.941 -07:00] [INF] [81] Jellyfin.Api.Helpers.MediaInfoHelper: User policy for "*****". EnablePlaybackRemuxing: True EnableVideoPlaybackTranscoding: True EnableAudioPlaybackTranscoding: True
[2024-11-08 11:36:21.941 -07:00] [INF] [81] Jellyfin.Api.Helpers.MediaInfoHelper: RemoteClientBitrateLimit: 10000000, RemoteIP: "***.***.***.***", IsInLocalNetwork: True

EDIT: There is no ffmpeg log as the media is direct playing without any modification.

[bitmap]

I had a ton of issues getting jf-ffmpeg7 to work with the dovi_tool extracted HDR10 HEVC files.  Reverting to jf-ffmpeg6 "solved" the problem.  I spoke with the devs a bunch to try to work it out, but what ended up happening was Gnattu wrote and integrated a function directly into Jellyfin-ffmpeg to remove Dolby Vision.

https://github.com/jellyfin/jellyfin-ffm...g/v7.0.2-4
https://github.com/jellyfin/jellyfin-ffmpeg/pull/480

As long as you are on version 7.0.2-4 and newer, you don't have to bother with dovi_tool.  I updated my DV to HDR10 guide to account for this.

https://forum.jellyfin.org/t-converting-...n-to-hdr10

Bonus feature, if you encode to AV1 with 7.0.2-4+ it will automatically strip the Dolby Vision EL and RPU for profiles 7 and 8.  If you did want to just remove DV, the command specified in my guide would be the way to do it with the bitstream filter.

Code:

-bsf:v hevc_metadata=remove_dovi=1

But with your ffmpeg command, you shouldn't need to bother with BL.HEVC.  Just feed in the original video and adjust your mapping.

I'll give this a shot. I need to check on my jellyfin-ffmpeg version. It should be the latest release, but I will do my dilligence. Thanks for posting info and a guide, I'll revise my encoding process accordingly if I can verify it works. 

For my understanding: I was under the impression that BL.hevc was the video stream with DV stripped when dovi_tool was run with the remove command, is that not the case? Did I miss in the dovi_tool documentation that it strips/modifies the original file when used in the fashion I've described? Reading through, it seemed like BL.hevc was the file to mux back in (or encode) once the process had been completed.

[TheDreadPirate]

The process you are using with dovi_tool is essentially the same as what I was doing. And in terms of removing Dolby Vision, the "remove" option removed both the RPU and EL, AFAIK. Same as what jf-ffmpeg7 is doing now.

There is just something in upstream ffmpeg7 that just didn't work with the files that dovi_tool produced or something. I tried remuxing just the HEVC file in MKV and MP4 before trying to merge it with the original audio/subs, etc. It just didn't work right.

But the new addition to jf-ffmpeg7.0.2-4 negates the need for dovi_tool for our use case.

[bitmap]

Perfect. I'll head down that path. Updated jellyfin-ffmpeg7 to 7.0.2-5-noble, which should incorporate the bitstream filter you've mentioned.

[gnattu]

This is the native HLS player of WebOS struggling which is not new. This is irrelevant if you remove dovi or not because they are removed after transcoding anyway.

[Host-in-the-Shell]

Do you by any chance agreed to everything under the terms of service for the TV? I had terrible performance issues on the JF app, manifesting itself similar to what you describe, until I deselected everything under the terms agreements and only kept the required terms. Turns out selecting all of that stuff makes your TV eat through a lot of memory and is specially noticeable during 4K HDR Playback in which audio is being transcoded or if you are using complex ASS subtitles, at least that was my experience.

You may also want to unplug the TV for around 5 mins to clear out some memory just in case.

[TheDreadPirate]

Does agreeing to the optional terms trigger WebOS to run a bunch of telemetry and monitoring software or something?

[Host-in-the-Shell]

I believe so. I had very similar issues as OP described, and after a while of trying to get these files playing, the app would crash with some message about running out of memory and that it would be restarted. After searching online for people experiencing something similar, I found somebody on reddit saying they turned off all those agreements and it helped them. I tried it and all the issues were gone after that. I ticked off everything except terms of use and privacy policy, and for my tv the agreements were located in support > privacy and terms > user agreements.