Converting Dolby Vision to HDR10

[TheDreadPirate]

This is going to sound stupid.  The solution is to remux the remux.

Code:

/usr/lib/jellyfin-ffmpeg/ffmpeg -hide_banner -loglevel error -stats -y -fflags +genpts+igndts -i "new_hdr10_video.mkv" -map 0 -c:v copy -c:a copy -c:s copy -max_muxing_queue_size 2048 -avoid_negative_ts make_zero -max_interleave_delta 0 "new_new_hdr10_video.mkv"

I will investigate if there is something I've missed, but this works for now.

[NavicNick]

Unfortunately that didn't work for me.

I did come across a program called MKVToolNix, and by using it's Multiplexer mode, I was able to select the video stream from the final file (before the second remux, so "new_HDR10_video.mkv") and the audio and subtitle files from the original MKV and remux them into a new file, and this file plays perfectly smooth everywhere with no transcoding needed.

Not sure why Jellyfin doesn't seem to like the files coming from FFMPEG, or why the second remux worked for you but not for me.

[NavicNick]

Just adding some more info on this program, because I found a way to fix my problem while also removing some steps from your process, but it does require another program.

Basically, I use your first command to make the .HEVC file for the video.
Then, I use MKVToolNix and the Multiplexer mode to add both the new .HEVC file and the Original MKV file as an input. From there, I deselect the video from the Original MKV, and select the video from the .HEVC, while leaving everything else in the original MKV selected (Subtitles, Audio, Chapters).
Finally, I click the "start multiplexing button" and once that is done I have a new MKV file with the HDR10 video from the HEVC file, and the audio, subtitles, and chapters from the Original MKV file.
This is what everything looks like in the MKVToolNix GUI. I left everything else default, didn't touch anything in the Output tab as it didn't seem to do anything for what I needed.

This gets rid of the Dolby Vision layers and solves my stuttering video problem, while removing a step from your guide (moving the .HEVC file into the MP4 container) and replacing the final remuxing step (ignoring the remux the remux step) with the Multiplexer in MKVToolNix.

[TheDreadPirate]

@NavicNick - Can you try the slightly tweaked commands in the opening post? I changed avoid_negative_ts to disabled and that appears to have resolved the issues with needing to mux twice originally.