External audio files out-of-sync with video

[jonathanweber]

Hello!

In order to better process what gets streamed to the client, I wanted to approach it this way:

• Extract the original audio tracks (DTS, AAC, ...) into external files from the original MKV (and named them according to the pattern in the documentation)
  
• Remux original MKV to remove audio tracks there
  
• Create multiple smaller versions of the soundless MKV (1080p, 720p, ...)
  
• Hard-Link the above audio-tracks for all smaller versions, so that Jellyfin detects them for all movie versions
  

It looks like this:

My Movie (2025) - Original.mkv (no audio tracks)
My Movie (2025) - Original.aac.eng.mka
My Movie (2025) - Original.dts.eng.mka

My Movie (2025) - 1080p.mkv (no audio tracks)
My Movie (2025) - 1080p.aac.eng.mka (Hardlink to original aac)
My Movie (2025) - 1080p.dts.eng.mka (Hardlink to original dts)
.....


In theory, this works like a charm. With every video quality version, I can select any audio track type I want, without re-encoding them with every version over and over again and wasting space.
Unfortunately, whenever I play a video with an external audio track, the video is out of sync by various amounts (but not drifting over time, it is just random between ~100ms to 3-5s).

I disabled video-reencode for me as the user, but allowed audio transcoding. Playback info in the webbrowser-version shows that video just gets passed through while audio gets transcoded to AAC.

For debugging, I tried to merge both an mkv and an audio stream again and the output file plays just fine using vlc. So I guess it might not be a problem with the video and audio files themselves.

Is this a known problem when Jellyfin is remuxing the video and audio stream during playback? Is there something I can do about it (without loosing the benefits of this setup)?

I am running Jellyfin 10.10.7 in Armbian / Ubuntu. The problem is present in any playback client I have tested: Android App and Web-Version.

Thanks in advance and best regards!
Jonathan