Audio Transcode switches off HDR playback

[dthree]

I have a number of 4k files with AV1 encoding and OPUS audio for English, AC3 for Spanish, Italian, and French dubs. When playing the file with the AC3 audio tracks, the playback is direct and the TV engages HDR mode. Switching to the OPUS audio causes the audio to transcode, which would be fine, except that HDR is no longer recognized by the TV and the image is flat and washed-out. The video is not being transcoded and subtitles are off. Somehow JF is not telling the TV its HDR.

I have Jellyfin 10.9.10 running in TrueNAS Scale, Dragonfish-24.04.2
Intel i5-12400f, 32G mem
There is an Intel ARC video card handling HW encode/decode.
AV1 or HEVC 4k 10bit playback is solid with the HW accel.

I did find an interesting bit in the logs.

When playing the stream with AC3 audio, the log shows this call for playback, Check out "setparams=color_primaries"

Code:

[2024-08-27 12:43:45.577 -04:00] [INF] [54] MediaBrowser.MediaEncoding.Transcoding.TranscodeManager: "/usr/lib/jellyfin-ffmpeg/ffmpeg" "-analyzeduration 200M -probesize 1G -init_hw_device vaapi=va:,kernel_driver=i915,driver=iHD -init_hw_device qsv=qs@va -init_hw_device opencl=ocl@va -filter_hw_device qs -hwaccel qsv -hwaccel_output_format qsv -c:v av1_qsv -noautorotate -i file:\"/mnt/path/Video/Movies/movie.mkv\" -noautoscale -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:1 -map -0:s -codec:v:0 av1_qsv -preset veryfast -b:v 11546608 -maxrate 11546608 -bufsize 23093216 -profile:v:0 main -level 50 -g:v:0 72 -keyint_min:v:0 72 -vf \"setparams=color_primaries=bt2020:color_trc=smpte2084:colorspace=bt2020nc,hwmap=derive_device=opencl,tonemap_opencl=format=nv12:p=bt709:t=bt709:m=bt709:tonemap=bt2390:peak=100:desat=0,hwmap=derive_device=qsv:reverse=1:extra_hw_frames=16,format=qsv\" -codec:a:0 libopus -strict -2 -ac 2 -ab 192000 -ar 48000 -copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 -f hls -max_delay 5000000 -hls_time 3 -hls_segment_type fmp4 -hls_fmp4_init_filename \"a4dcf19d58cf3cebae11adae395497ed-1.mp4\" -start_number 0 -hls_segment_filename \"/cache/transcodes/a4dcf19d58cf3cebae11adae395497ed%d.mp4\" -hls_playlist_type vod -hls_list_size 0 -y \"/cache/transcodes/a4dcf19d58cf3cebae11adae395497ed.m3u8\""

But when playing with the transcoded OPUS audio, it looks like this:

Code:

[2024-08-27 12:36:19.894 -04:00] [INF] [63] MediaBrowser.MediaEncoding.Transcoding.TranscodeManager: "/usr/lib/jellyfin-ffmpeg/ffmpeg" "-analyzeduration 200M -probesize 1G -init_hw_device vaapi=va:,kernel_driver=i915,driver=iHD -init_hw_device qsv=qs@va -filter_hw_device qs -hwaccel qsv -hwaccel_output_format qsv -c:v av1_qsv -noautorotate -i file:\"/mnt/path/Video/Movies/movie.mkv\" -noautoscale -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:1 -map -0:s -codec:v:0 av1_qsv -preset veryfast -b:v 11546608 -maxrate 11546608 -bufsize 23093216 -profile:v:0 main -level 50 -g:v:0 72 -keyint_min:v:0 72 -vf \"setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale_qsv=format=nv12\" -codec:a:0 libopus -strict -2 -ac 2 -ab 192000 -ar 48000 -copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 -f hls -max_delay 5000000 -hls_time 3 -hls_segment_type fmp4 -hls_fmp4_init_filename \"9552872040325d6da98ca2fb8e0ba090-1.mp4\" -start_number 0 -hls_segment_filename \"/cache/transcodes/9552872040325d6da98ca2fb8e0ba090%d.mp4\" -hls_playlist_type vod -hls_list_size 0 -y \"/cache/transcodes/9552872040325d6da98ca2fb8e0ba090.m3u8\""

Why is the color_primaries param set for bt709 instead of bt2020 when the only difference is OPUS vs AC3?
My workaround has been to just transcode to a new file with AC3 but that's a lot of manual work for something JF was built to do.

I did see a similar bug report from 2022, but it was marked "completed" and nothing since.

[TheDreadPirate]

Both are transcoding the video, both being converted to SDR, only the first one is tone mapping.

Can you share the media info for each file? And what client are you playing these videos on?

[dthree]

Small correction, it is only transcoding 7.1 OPUS, but 5.1 OPUS is playing directly. Effect is the same, only plays in HDR when not transcoding audio. Also, the looked at the logs after testing this again and found that there is no transcode log when playing this file with the 5.1 audio as it is playing directly, so that log snippet above for AC3 playback is incorrect.
This is playing in the Jellyfin app on a Roku 4802X

Here is the media info.

Code:

General
Unique ID                                : 83441409859904805117689112558248275385 (0x3EC63B21D70C4CC26EBF9CB24A7A2DB9)
Complete name                            : F:\Movies\Movie.mkv
Format                                  : Matroska
Format version                          : Version 4
File size                                : 9.60 GiB
Duration                                : 1 h 58 min
Overall bit rate                        : 11.5 Mb/s
Frame rate                              : 23.976 FPS
Encoded date                            : 2024-05-27 13:22:13 UTC
Writing application                      : mkvmerge v84.0 ('Sleeper') 64-bit
Writing library                          : libebml v1.4.5 + libmatroska v1.7.1 / Lavf60.16.100

Video
ID                                      : 1
Format                                  : AV1
Format/Info                              : AOMedia Video 1
Format profile                          : Main@L5.0
HDR format                              : SMPTE ST 2086, HDR10 compatible
Codec ID                                : V_AV1
Duration                                : 1 h 58 min
Bit rate                                : 10.4 Mb/s
Width                                    : 3 840 pixels
Height                                  : 1 604 pixels
Display aspect ratio                    : 2.39:1
Frame rate mode                          : Constant
Frame rate                              : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                      : 4:2:0 (Type 2)
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                      : 0.070
Stream size                              : 8.64 GiB (90%)
Default                                  : Yes
Forced                                  : No
Color range                              : Limited
Color primaries                          : BT.2020
Transfer characteristics                : PQ
Matrix coefficients                      : BT.2020 non-constant
Mastering display color primaries        : Display P3
Mastering display luminance              : min: 0.0001 cd/m2, max: 1000 cd/m2
Maximum Content Light Level              : 1000
MaxCLL_Original                          : 1000 cd/m2
Maximum Frame-Average Light Level        : 168
MaxFALL_Original                        : 168 cd/m2

Audio #1
ID                                      : 2
Format                                  : Opus
Codec ID                                : A_OPUS
Duration                                : 1 h 58 min
Bit rate                                : 478 kb/s
Channel(s)                              : 8 channels
Channel layout                          : L R C Ls Rs Lb Rb LFE
Sampling rate                            : 48.0 kHz
Frame rate                              : 50.000 FPS (960 SPF)
Compression mode                        : Lossy
Stream size                              : 407 MiB (4%)
Language                                : English
Default                                  : Yes
Forced                                  : No

Audio #2
ID                                      : 3
Format                                  : Opus
Codec ID                                : A_OPUS
Duration                                : 1 h 58 min
Bit rate                                : 315 kb/s
Channel(s)                              : 6 channels
Channel layout                          : L R C Lb Rb LFE
Sampling rate                            : 48.0 kHz
Frame rate                              : 50.000 FPS (960 SPF)
Compression mode                        : Lossy
Stream size                              : 269 MiB (3%)
Language                                : French
Default                                  : No
Forced                                  : No

Audio #3
ID                                      : 4
Format                                  : Opus
Codec ID                                : A_OPUS
Duration                                : 1 h 58 min
Bit rate                                : 315 kb/s
Channel(s)                              : 6 channels
Channel layout                          : L R C Lb Rb LFE
Sampling rate                            : 48.0 kHz
Frame rate                              : 50.000 FPS (960 SPF)
Compression mode                        : Lossy
Stream size                              : 268 MiB (3%)
Language                                : Spanish
Default                                  : No
Forced                                  : No

Text #1
ID                                      : 5
Format                                  : PGS
Muxing mode                              : zlib
Codec ID                                : S_HDMV/PGS
Codec ID/Info                            : Picture based subtitle format used on BDs/HD-DVDs
Duration                                : 1 h 45 min
Bit rate                                : 28.0 kb/s
Frame rate                              : 0.347 FPS
Count of elements                        : 2202
Stream size                              : 21.2 MiB (0%)
Language                                : English
Default                                  : No
Forced                                  : No

Text #2
ID                                      : 6
Format                                  : PGS
Muxing mode                              : zlib
Codec ID                                : S_HDMV/PGS
Codec ID/Info                            : Picture based subtitle format used on BDs/HD-DVDs
Duration                                : 1 h 57 min
Bit rate                                : 22.7 kb/s
Frame rate                              : 0.267 FPS
Count of elements                        : 1888
Stream size                              : 19.1 MiB (0%)
Language                                : French
Default                                  : No
Forced                                  : No

Text #3
ID                                      : 7
Format                                  : PGS
Muxing mode                              : zlib
Codec ID                                : S_HDMV/PGS
Codec ID/Info                            : Picture based subtitle format used on BDs/HD-DVDs
Duration                                : 1 h 57 min
Bit rate                                : 21.3 kb/s
Frame rate                              : 0.286 FPS
Count of elements                        : 2022
Stream size                              : 17.9 MiB (0%)
Language                                : Spanish
Default                                  : No
Forced                                  : No

[TheDreadPirate]

What I'm seeing from Roku's docs, and various Reddit posts for Plex/Jellyfin/Emby on Roku, seems to indicate that Roku only supports 7.1 audio for Dolby Digital and Dolby Digital+. DTS is passed through. For everything else Roku's docs recommend AAC stereo, but do state they support OPUS in MKV but don't state how many channels. But the Reddit posts seem to indicate that 5.1 is the cap for non-Dolby codecs.

[dthree]

I get why it needs to transcode the audio for 7.1, and I can confirm it does this with AAC 7.1 files as well as OPUS 7.1 audio. The issue is that it should still direct play the video (Direct Stream as it's called in the docs) while transcoding just the audio, but it's not. When playing Dolby TruHD 7.1 files, the player correctly transcodes the audio to 5.1, and passes through the video, retaining the HDR. Here's an example of that.

Code:

[2024-08-29 15:20:08.082 -04:00] [INF] [49] MediaBrowser.MediaEncoding.Transcoding.TranscodeManager: "/usr/lib/jellyfin-ffmpeg/ffmpeg" "-analyzeduration 200M -probesize 1G -readrate 10 -fflags +genpts  -i file:\"/mnt/Movies/TrueHD_movie.mkv\" -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:1 -map -0:s -codec:v:0 copy -tag:v:0 hvc1 -bsf:v hevc_mp4toannexb -start_at_zero -codec:a:0 libfdk_aac -ac 6 -ab 640000 -copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 -f hls -max_delay 5000000 -hls_time 6 -hls_segment_type mpegts -start_number 0 -hls_segment_filename \"/cache/transcodes/6fd59b8ffbe8b93c5cf1996370358599%d.ts\" -hls_playlist_type vod -hls_list_size 0 -y \"/cache/transcodes/6fd59b8ffbe8b93c5cf1996370358599.m3u8\""

You can see it's copying the video track and transcoding the audio track to aac 6 channel. I'm not sure why this isn't the behavior for OPUS or AAC 7.1 audio tracks.

[TheDreadPirate]

I am pretty sure my Roku Ultra is also a 4802X (I only use it for testing and troubleshooting).  Most of my media is AV1 + some variety of Dolby or DTS audio.  I also have test instance of Jellyfin with some experimental media with OPUS and FLAC audio.

I will do some testing when I get home.  I only learned just now that Roku supports AV1 and that the chipset in the 4802X (Realtek RTD1319) supports AV1 decoding.