Converting Dolby Vision to HDR10

[TheDreadPirate]

****This guide has been updated****

Jellyfin-ffmpeg 7.0.2-4 added a feature to remove Dolby Vision directly. The process of removing Dolby Vision can now be accomplished with a single ffmpeg command, greatly simplifying the process. I am leaving the original walkthrough for those on non-Jellyfin builds of ffmpeg or jellyfin-ffmpeg6.

Code:

/usr/lib/jellyfin-ffmpeg/ffmpeg -y -hide_banner -stats -fflags +genpts+igndts -loglevel error -i "/path/to/dv_video.mkv" -map 0 -bsf:v hevc_metadata=remove_dovi=1 -codec copy -max_muxing_queue_size 2048 -max_interleave_delta 0 -avoid_negative_ts disabled hdr10_video.mkv

And that is it. Still no re-encoding required.

A bonus feature: If you do re-encode, to AV1 specifically, you can skip using the bitstream filter entirely. Re-encoding to AV1 will automatically strip the Dolby Vision EL and RPU while maintaining the BT2020 HDR10 color space.

=========================================================================

****Original Walkthrough****

If you've watched HDR content on Jellyfin you've likely struggled with the mish-mash of client/server compatibility that comes with HDR content.  Especially with Dolby Vision.

In my quest to encode my entire library with AV1, that required I tackle the issue of Dolby Vision.  Both because HDR10 is vastly more compatible (yay open standards) and because AV1 does not officially support these Dolby Vision profiles anyway (AV1 Dolby Vision via Profile 10).  To be clear, you can still use HEVC if you haven't gotten on the AV1 bandwagon yet.  We're going to focus on simply removing Dolby Vision, which does not require any re-encoding.

For this guide we are focusing on the two more common Dolby Vision profiles: 7.6 and 8.1.  Profile 7.6 is used exclusively by 4K HDR Blurays and can only be decoded by Bluray players.  To my knowledge, there are no mobile phones or streaming devices that support profile 7.6.  Then there's the related profile 8.1.

Both Dolby Vision profiles use HDR10 as their base and add dynamic metadata, a.k.a. enhancement layers, on top of it.  A single layer, called the RPU (Reference Picture Unit), for profile 8.1.  And two enhancement layers for profile 7.6, RPU and MEL/FEL (minimum or full enhancement layer).  Converting Dolby Vision 7.6 and 8.1 to HDR10 simply requires discarding these extra layers.

FFMPEG cannot, by itself, discard Dolby Vision metadata.  We will need the aid of an open source app called dovi_tool.  I've tested this process in Linux, but both dovi_tool and jellyfin-ffmpeg6 are available on Windows and Mac.  You may need to make minor syntax adjustments for Windows and Mac.  Vanilla FFMPEG should also work fine.

First we need extract the video and run it through dovi_tool to remove the Dolby Vision layers.

Code:

/usr/lib/jellyfin-ffmpeg/ffmpeg -y -hide_banner -loglevel error -stats -i "/path/to/your/DV_video.mkv" -dn -c:v copy -bsf:v hevc_mp4toannexb -f hevc - | dovi_tool remove - -o "hdr10_video.hevc"

Now we have a naked HDR10 HEVC file with no structure.  I tried directly remuxing the video back into the source MKV file, but muxing it into a MP4 container first seemed to work better.

Code:

/usr/lib/jellyfin-ffmpeg/ffmpeg -hide_banner -loglevel error -stats -y -fflags +genpts -i "hdr10_video.hevc" -c:v copy "hdr10_video.mp4"

NOW we can remux the new HDR10 video with the original audio and subtitles.

Code:

/usr/lib/jellyfin-ffmpeg/ffmpeg -hide_banner -loglevel error -stats -y -fflags +genpts+igndts -i "hdr10_video.mp4" -i "/path/to/your/DV_video.mkv" -map_chapters 1 -map 0:v -map 1:a -map 1:s -c:v copy -c:a copy -c:s copy -max_muxing_queue_size 2048 -avoid_negative_ts disabled -max_interleave_delta 0 "new_hdr10_video.mkv"

Now we've muxed the HDR10 video with the original audio, subtitles, and chapter markers.  Zero re-encoding required.

[NavicNick]

Hey again, thanks for posting this!

According to Jellyfin's Media Info tab on the movie I did this to, it worked. It got rid of the Dolby Vision layers and is now just HDR10. But now I'm having some other weirdness.

Basically, Jellyfin won't read the final file at all, it just stays on a loading screen. This happens in the web app, the android app, and findroid (findroid doesnt show a loading screen, it just gets stuck on the play/fast forward UI and greys out some buttons).
My NAS also has it's RAM eaten up entirely by Jellyfin (16gb) and the CPU spikes in usage every now and then to 50% or more. Even after I close the window, the CPU will stay around 25% usage and the RAM usage will increase, until whatever process Jellyfin is trying to do seemingly fails, freeing up the RAM, and then restarts. This CPU usage behavior only goes away when I terminate the Jellyfin app in Truenas and start it again.

VLC also won't play the final file instantly, it just reads the file from my NAS (or external SSD if I put the file on there) at 385mbps. That is, until I click a point on the timeline, then it plays the video just fine, but there is no audio. Even though there are many different audio streams I can select, and Jellyfin as well as potplayer identify them, none of them seem to output any audio. Similarly, subtitles don't seem to work either, even though they are identified.

I suspect there is something that went wrong going from the MP4 file to the final file, as the MP4 file (the file made from taking the HEVC file and putting it into the MP4 container) plays instantly and as expected in VLC. I did try and upload this file (the HEVC to MP4 file) to Jellyfin and it stuttered a ton (unless it was transcoded to 1080p 20mbps, anything higher had slight stuttering), even though it played fine in VLC. Probably because there is no audio? But just adding info in case it helps.

Just in case, here is the commands I ran (I'm running Windows 11, FFMPEG for Jellyfin 6.0.1-8-portable_win64, DoVi 2.1.2, also tried with normal FFMPEG)

Code:

ffmpeg -y -hide_banner -loglevel error -stats -i "D:\Media\The Batman.mkv" -dn -c:v copy -vbsf hevc_mp4toannexb -f hevc - | dovi_tool remove - -o "F:\FFMPEG\hdr10_video.hevc"

Code:

ffmpeg -hide_banner -loglevel error -stats -y -fflags +genpts -i "F:\FFMPEG\hdr10_video.hevc" -c:v copy "D:\Media\hdr10_video.mp4"

Code:

ffmpeg -hide_banner -loglevel error -stats -y -fflags +genpts -i "D:\Media\hdr10_video.mp4" -i "D:\Media\The Batman.mkv" -map_chapters 1 -map 0:v -map 1:a -map 1:s -c:v copy -c:a copy -c:s copy "R:\FFMPEG\The Batman.mkv"

[TheDreadPirate]

If you still have the MP4, try adding these options when remuxing for the final MKV.

Code:

-max_muxing_queue_size 2048 -avoid_negative_ts make_zero -max_interleave_delta 0

You can just add them to the end of the ffmpeg command before the output file.

Also try changing

Code:

+genpts

to

Code:

+genpts+igndts

If this works for you, I will update the original post.

[NavicNick]

So close to being perfect!

When I open this new file in VLC, it plays back as expected. But when I play it through Jellyfin, there is some minor stuttering on Findroid, and massive stuttering through my browser on my PC. I can send screen recordings if you want.
Jellyfin Media Player on my PC plays back smoothly, but not in HDR even though it is direct play (different problem I think, happens on HDR10 movies as well), and the Jellyfin android app also plays back smoothly, but is transcoding the media.

Good news is that the movie does play through Jellyfin, so your tweaks did fix that problem, but I'm not sure where this stuttering problem could be coming from. It only happens when the movie is direct playing, and does not happen when it is being transcoded.

[TheDreadPirate]

Can you share the media info, as displayed in Jellyfin's UI, for the new file?

[NavicNick]

Yeah, here you go:

Code:

The Batman
Container: mkv
Path: /SMB/Movies/The Batman (2022)/The Batman.mkv
Size: 82300 MB

Video
Title: 4K HEVC HDR
Codec: HEVC
AVC: No
Profile: Main 10
Level: 153
Resolution: 3840x2160
Aspect ratio: 16:9
Anamorphic: No
Interlaced: No
Framerate: 23.976215
Bitrate: 65304 kbps
Bit depth: 10 bit
Video range: HDR
Video range type: HDR10
Color space: bt2020nc
Color transfer: smpte2084
Color primaries: bt2020
Pixel format: yuv420p10le
Ref frames: 1

Audio
Title: Surround 7.1 - English - TRUEHD - Default
Language: eng
Codec: TRUEHD
AVC: No
Profile: Dolby TrueHD + Dolby Atmos
Layout: 7.1
Channels: 8 ch
Bitrate: 3028 kbps
Sample rate: 48000 Hz
Bit depth: 24 bit
Default: Yes
Forced: No
External: No

Subtitle
Title: English - Default - PGSSUB
Language: eng
Codec: PGSSUB
AVC: No
Resolution: 1920x1080
Default: Yes
Forced: No
External: No

Audio
Title: Surround 5.1 - English - Dolby Digital
Language: eng
Codec: AC3
AVC: No
Layout: 5.1
Channels: 6 ch
Bitrate: 448 kbps
Sample rate: 48000 Hz
Default: No
Forced: No
External: No

Audio
Title: Surround 5.1 - English - Dolby Digital
Language: eng
Codec: AC3
AVC: No
Layout: 5.1
Channels: 6 ch
Bitrate: 448 kbps
Sample rate: 48000 Hz
Default: No
Forced: No
External: No

Audio
Title: Surround 5.1 - English - Dolby Digital
Language: eng
Codec: AC3
AVC: No
Layout: 5.1
Channels: 6 ch
Bitrate: 448 kbps
Sample rate: 48000 Hz
Default: No
Forced: No
External: No

Audio
Title: Surround 5.1 - English - Dolby Digital
Language: eng
Codec: AC3
AVC: No
Layout: 5.1
Channels: 6 ch
Bitrate: 448 kbps
Sample rate: 48000 Hz
Default: No
Forced: No
External: No

Audio
Title: Surround 5.1 - French - Dolby Digital
Language: fra
Codec: AC3
AVC: No
Layout: 5.1
Channels: 6 ch
Bitrate: 448 kbps
Sample rate: 48000 Hz
Default: No
Forced: No
External: No

Audio
Title: Surround 7.1 - German - TRUEHD
Language: deu
Codec: TRUEHD
AVC: No
Profile: Dolby TrueHD + Dolby Atmos
Layout: 7.1
Channels: 8 ch
Bitrate: 3064 kbps
Sample rate: 48000 Hz
Bit depth: 24 bit
Default: No
Forced: No
External: No

Audio
Title: Surround 5.1 - German - Dolby Digital
Language: deu
Codec: AC3
AVC: No
Layout: 5.1
Channels: 6 ch
Bitrate: 448 kbps
Sample rate: 48000 Hz
Default: No
Forced: No
External: No

Audio
Title: Surround 5.1 - German - Dolby Digital
Language: deu
Codec: AC3
AVC: No
Layout: 5.1
Channels: 6 ch
Bitrate: 448 kbps
Sample rate: 48000 Hz
Default: No
Forced: No
External: No

Audio
Title: Stereo - German - Dolby Digital
Language: deu
Codec: AC3
AVC: No
Layout: stereo
Channels: 2 ch
Bitrate: 192 kbps
Sample rate: 48000 Hz
Default: No
Forced: No
External: No

Audio
Title: Surround 7.1 - Italian - TRUEHD
Language: ita
Codec: TRUEHD
AVC: No
Profile: Dolby TrueHD + Dolby Atmos
Layout: 7.1
Channels: 8 ch
Bitrate: 3042 kbps
Sample rate: 48000 Hz
Bit depth: 24 bit
Default: No
Forced: No
External: No

Audio
Title: Surround 5.1 - Italian - Dolby Digital
Language: ita
Codec: AC3
AVC: No
Layout: 5.1
Channels: 6 ch
Bitrate: 448 kbps
Sample rate: 48000 Hz
Default: No
Forced: No
External: No

Audio
Title: Surround 5.1 - Italian - Dolby Digital
Language: ita
Codec: AC3
AVC: No
Layout: 5.1
Channels: 6 ch
Bitrate: 448 kbps
Sample rate: 48000 Hz
Default: No
Forced: No
External: No

Audio
Title: Surround 5.1 - Spanish - Dolby Digital
Language: spa
Codec: AC3
AVC: No
Layout: 5.1
Channels: 6 ch
Bitrate: 448 kbps
Sample rate: 48000 Hz
Default: No
Forced: No
External: No

Subtitle
Title: French - PGSSUB
Language: fra
Codec: PGSSUB
AVC: No
Resolution: 1920x1080
Default: No
Forced: No
External: No

Subtitle
Title: German - PGSSUB
Language: deu
Codec: PGSSUB
AVC: No
Resolution: 1920x1080
Default: No
Forced: No
External: No

Subtitle
Title: Italian - PGSSUB
Language: ita
Codec: PGSSUB
AVC: No
Resolution: 1920x1080
Default: No
Forced: No
External: No

Subtitle
Title: Spanish - PGSSUB
Language: spa
Codec: PGSSUB
AVC: No
Resolution: 1920x1080
Default: No
Forced: No
External: No

Subtitle
Title: Danish - PGSSUB
Language: dan
Codec: PGSSUB
AVC: No
Resolution: 1920x1080
Default: No
Forced: No
External: No

Subtitle
Title: Finnish - PGSSUB
Language: fin
Codec: PGSSUB
AVC: No
Resolution: 1920x1080
Default: No
Forced: No
External: No

Subtitle
Title: Norwegian - PGSSUB
Language: nor
Codec: PGSSUB
AVC: No
Resolution: 1920x1080
Default: No
Forced: No
External: No

Subtitle
Title: Swedish - PGSSUB
Language: swe
Codec: PGSSUB
AVC: No
Resolution: 1920x1080
Default: No
Forced: No
External: No

Subtitle
Title: French - PGSSUB
Language: fra
Codec: PGSSUB
AVC: No
Default: No
Forced: No
External: No

Subtitle
Title: German - PGSSUB
Language: deu
Codec: PGSSUB
AVC: No
Default: No
Forced: No
External: No

Subtitle
Title: Italian - PGSSUB
Language: ita
Codec: PGSSUB
AVC: No
Default: No
Forced: No
External: No

Subtitle
Title: Spanish - PGSSUB
Language: spa
Codec: PGSSUB
AVC: No
Default: No
Forced: No
External: No

Edit: Just made the copy-paste into a code block so it takes up less space in the thread.

[TheDreadPirate]

What happens if you make both of these changes both when making the new HDR10 MP4 and the final MKV?

Code:

-max_muxing_queue_size 2048 -avoid_negative_ts make_zero -max_interleave_delta 0

Code:

+genpts+igndts

Code:

/usr/lib/jellyfin-ffmpeg/ffmpeg -hide_banner -loglevel error -stats -y -fflags +genpts+igndts -i "hdr10_video.hevc" -c:v copy  -max_muxing_queue_size 2048 -avoid_negative_ts make_zero -max_interleave_delta 0 "hdr10_video.mp4"

Code:

/usr/lib/jellyfin-ffmpeg/ffmpeg -hide_banner -loglevel error -stats -y -fflags +genpts+igndts -i "hd10_video.mp4" -i "/path/to/your/DV_video.mkv" -map_chapters 1 -map 0:v -map 1:a -map 1:s -c:v copy -c:a copy -c:s copy -max_muxing_queue_size 2048 -avoid_negative_ts make_zero -max_interleave_delta 0 "new_hdr10_video.mkv"

[NavicNick]

Same result, still massive stutters in browser and slight stutters on Findroid, but transcoding results in properly smooth playback (without HDR because of transcoding).

Just in case, here are the commands I ran:

Code:

ffmpeg -y -hide_banner -loglevel error -stats -i "F:\The Batman.mkv" -dn -c:v copy -bsf:v hevc_mp4toannexb -f hevc - | dovi_tool remove - -o "D:\FFMPEG\1-HEVC\hdr10_video.hevc"

Code:

ffmpeg -hide_banner -loglevel error -stats -y -fflags +genpts+igndts -i "D:\FFMPEG\1-HEVC\hdr10_video.hevc" -c:v copy -max_muxing_queue_size 2048 -avoid_negative_ts make_zero -max_interleave_delta 0 "D:\FFMPEG\2-MP4\hdr10_video.mp4"

Code:

ffmpeg -hide_banner -loglevel error -stats -y -fflags +genpts+igndts -i "D:\FFMPEG\2-MP4\hdr10_video.mp4" -i "F:\The Batman.mkv" -map_chapters 1 -map 0:v -map 1:a -map 1:s -c:v copy -c:a copy -c:s copy -max_muxing_queue_size 2048 -avoid_negative_ts make_zero -max_interleave_delta 0 "D:\FFMPEG\3-FINAL\new_hdr10_video.mkv"

I can try tomorrow with a different movie (Dune: Part Two) to see if it's just something with The Batman (which is what I've been using as my Dolby Vision test case this whole time) but ideally we'd figure this problem out.
EDIT: Tried this with Dune Part Two and had the same result. So it's not a movie specific issue.

[TheDreadPirate]

I'll have to do some testing and get back to you. These steps were derived from my AV1 encode process, minus the re-encoding. I'll try doing a true replication of this process.

[TheDreadPirate]

Huh....Ok. I was able to reproduce. Now to figure out how to fix it without having to re-encode.