2025-02-27, 07:30 AM
Well, I've made the horrible mistake of choosing to go down this rabbit hole, but I have new findings that might end up turning this into a pull request (if the root cause can actually be found). I just need a bit of help! I was able to get the WebOS client play my problem videos as a Remux (not even falling back to direct stream!) by forcing direct stream and allow video stream copy to true in the web client. Add the following lines to here to replicate:
I don't fully understand the implications of making this request, for example, if a video truly cannot direct stream, how the backend handles choosing the best alternative, or if it just fails.
This is what I've gathered so far:
1. For reasons I don't understand, the direct play fails with a "DirectPlayError" reason. From my research, this only happens when there's a bug in Jellyfin OR the browser advertises a codec in its profile that it doesn't actually support. This triggers the error handler, which has an initial value for EnableDirectStream and AllowVideoStreamCopy. These two are both set to the result of "isRemoteSource && !isAlreadyFallbacking". I'm guessing that at this stage, the result is FALSE, and my logs confirmed that. A change stream call is made.
2. changeStream calls getPlaybackInfo, nothing weird here.
3. In getPlaybackInfo, another way for the EnableDirectStream to be false. Request is kicked off to backend.
4. The POST handler calls SetDeviceSpecificData.
5. This is the section of code that makes me think options.EnableDirectStream is never actually used (and I don't see it used in the generation of the client video URL)...
First of all, direct streaming via http seems to work, what do they even mean? And secondly, ForceDirectStream is never populated by the backend OR jellyfin-web, so in EVERY case, EnableDirectStream is false (hence why I think the variable is unused).
6. Another interesting one in the same function. A "mediaSource" directstream boolean is separately set (presumably) to false due to the options variable being false. But...
7. Just a bit later, there's a check to append "allowVideoStreamCopy=false". Normally this would take place, but the earlier forced value allowVideoStreamCopy being TRUE makes this not take place.
8. The client takes in this slightly modified URL (now without the "allowVideoStreamCopy=false" ) and makes a request to get the best video feed. Because copying is now allowed, the best transcode profile is a copy.
9. Finally, a call to TryStreamCopy. Miraculously, CanStreamCopyVideo returns true. And somehow, and this is the part I *really* don't understand, I get a remux that seems to just be two direct streams. WHAT?
Here is the command that was given to remux with ffmpeg:
Code:
query.EnableDirectStream = true;
query.AllowVideoStreamCopy = true;I don't fully understand the implications of making this request, for example, if a video truly cannot direct stream, how the backend handles choosing the best alternative, or if it just fails.
This is what I've gathered so far:
1. For reasons I don't understand, the direct play fails with a "DirectPlayError" reason. From my research, this only happens when there's a bug in Jellyfin OR the browser advertises a codec in its profile that it doesn't actually support. This triggers the error handler, which has an initial value for EnableDirectStream and AllowVideoStreamCopy. These two are both set to the result of "isRemoteSource && !isAlreadyFallbacking". I'm guessing that at this stage, the result is FALSE, and my logs confirmed that. A change stream call is made.
2. changeStream calls getPlaybackInfo, nothing weird here.
3. In getPlaybackInfo, another way for the EnableDirectStream to be false. Request is kicked off to backend.
4. The POST handler calls SetDeviceSpecificData.
5. This is the section of code that makes me think options.EnableDirectStream is never actually used (and I don't see it used in the generation of the client video URL)...
Code:
if (!options.ForceDirectStream)
{
// direct-stream http streaming is currently broken
options.EnableDirectStream = false;
}6. Another interesting one in the same function. A "mediaSource" directstream boolean is separately set (presumably) to false due to the options variable being false. But...
7. Just a bit later, there's a check to append "allowVideoStreamCopy=false". Normally this would take place, but the earlier forced value allowVideoStreamCopy being TRUE makes this not take place.
8. The client takes in this slightly modified URL (now without the "allowVideoStreamCopy=false" ) and makes a request to get the best video feed. Because copying is now allowed, the best transcode profile is a copy.
9. Finally, a call to TryStreamCopy. Miraculously, CanStreamCopyVideo returns true. And somehow, and this is the part I *really* don't understand, I get a remux that seems to just be two direct streams. WHAT?
Here is the command that was given to remux with ffmpeg:
Code:
{"Protocol":0,"Id":"1234","Path":"/data/Media/file.mkv","EncoderPath":null,"EncoderProtocol":null,"Type":0,"Container":"mkv","Size":7404401956,"Name":"Title","IsRemote":false,"ETag":"1234","RunTimeTicks":14411099999,"ReadAtNativeFramerate":false,"IgnoreDts":false,"IgnoreIndex":false,"GenPtsInput":false,"SupportsTranscoding":true,"SupportsDirectStream":true,"SupportsDirectPlay":true,"IsInfiniteStream":false,"UseMostCompatibleTranscodingProfile":false,"RequiresOpening":false,"OpenToken":null,"RequiresClosing":false,"LiveStreamId":null,"BufferMs":null,"RequiresLooping":false,"SupportsProbing":true,"VideoType":0,"IsoType":null,"Video3DFormat":null,"MediaStreams":[{"Codec":"h264","CodecTag":null,"Language":null,"ColorRange":null,"ColorSpace":null,"ColorTransfer":null,"ColorPrimaries":null,"DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Rotation":null,"Comment":null,"TimeBase":"1/1000","CodecTimeBase":null,"Title":null,"VideoRange":1,"VideoRangeType":1,"VideoDoViTitle":null,"AudioSpatialFormat":0,"LocalizedUndefined":null,"LocalizedDefault":null,"LocalizedForced":null,"LocalizedExternal":null,"LocalizedHearingImpaired":null,"DisplayTitle":"1080p H264 SDR","NalLengthSize":"4","IsInterlaced":false,"IsAVC":true,"ChannelLayout":null,"BitRate":41103882,"BitDepth":8,"RefFrames":1,"PacketLength":null,"Channels":null,"SampleRate":null,"IsDefault":true,"IsForced":false,"IsHearingImpaired":false,"Height":1080,"Width":1920,"AverageFrameRate":23.976025,"RealFrameRate":23.976025,"ReferenceFrameRate":23.976025,"Profile":"High","Type":1,"AspectRatio":"16:9","Index":0,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Path":null,"PixelFormat":"yuv420p","Level":41,"IsAnamorphic":false},{"Codec":"pcm_s24le","CodecTag":null,"Language":"eng","ColorRange":null,"ColorSpace":null,"ColorTransfer":null,"ColorPrimaries":null,"DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Rotation":null,"Comment":null,"TimeBase":"1/1000","CodecTimeBase":null,"Title":"Commentary","VideoRange":0,"VideoRangeType":0,"VideoDoViTitle":null,"AudioSpatialFormat":0,"LocalizedUndefined":null,"LocalizedDefault":"Default","LocalizedForced":null,"LocalizedExternal":"External","LocalizedHearingImpaired":null,"DisplayTitle":"Commentary","NalLengthSize":null,"IsInterlaced":false,"IsAVC":false,"ChannelLayout":null,"BitRate":2304000,"BitDepth":24,"RefFrames":null,"PacketLength":null,"Channels":2,"SampleRate":48000,"IsDefault":false,"IsForced":false,"IsHearingImpaired":false,"Height":null,"Width":null,"AverageFrameRate":null,"RealFrameRate":null,"ReferenceFrameRate":null,"Profile":null,"Type":0,"AspectRatio":null,"Index":1,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Path":null,"PixelFormat":null,"Level":0,"IsAnamorphic":null},{"Codec":"flac","CodecTag":null,"Language":"jpn","ColorRange":null,"ColorSpace":null,"ColorTransfer":null,"ColorPrimaries":null,"DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Rotation":null,"Comment":null,"TimeBase":"1/1000","CodecTimeBase":null,"Title":"Japanese / FLAC / 2.0 / 48 kHz / 1346 kbps","VideoRange":0,"VideoRangeType":0,"VideoDoViTitle":null,"AudioSpatialFormat":0,"LocalizedUndefined":null,"LocalizedDefault":"Default","LocalizedForced":null,"LocalizedExternal":"External","LocalizedHearingImpaired":null,"DisplayTitle":"Japanese / FLAC / 2.0 / 48 kHz / 1346 kbps","NalLengthSize":null,"IsInterlaced":false,"IsAVC":false,"ChannelLayout":"stereo","BitRate":1345952,"BitDepth":24,"RefFrames":null,"PacketLength":null,"Channels":2,"SampleRate":48000,"IsDefault":true,"IsForced":false,"IsHearingImpaired":false,"Height":null,"Width":null,"AverageFrameRate":null,"RealFrameRate":null,"ReferenceFrameRate":null,"Profile":null,"Type":0,"AspectRatio":null,"Index":2,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Path":null,"PixelFormat":null,"Level":0,"IsAnamorphic":null},{"Codec":"eac3","CodecTag":null,"Language":"eng","ColorRange":null,"ColorSpace":null,"ColorTransfer":null,"ColorPrimaries":null,"DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Rotation":null,"Comment":null,"TimeBase":"1/1000","CodecTimeBase":null,"Title":"English / E-AC-3 / 2.0 / 48 kHz / 224 kbps","VideoRange":0,"VideoRangeType":0,"VideoDoViTitle":null,"AudioSpatialFormat":0,"LocalizedUndefined":null,"LocalizedDefault":"Default","LocalizedForced":null,"LocalizedExternal":"External","LocalizedHearingImpaired":null,"DisplayTitle":"English / E-AC-3 / 2.0 / 48 kHz / 224 kbps","NalLengthSize":null,"IsInterlaced":false,"IsAVC":false,"ChannelLayout":"stereo","BitRate":224000,"BitDepth":null,"RefFrames":null,"PacketLength":null,"Channels":2,"SampleRate":48000,"IsDefault":false,"IsForced":false,"IsHearingImpaired":false,"Height":null,"Width":null,"AverageFrameRate":null,"RealFrameRate":null,"ReferenceFrameRate":null,"Profile":null,"Type":0,"AspectRatio":null,"Index":3,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Path":null,"PixelFormat":null,"Level":0,"IsAnamorphic":null},{"Codec":"ass","CodecTag":null,"Language":"eng","ColorRange":null,"ColorSpace":null,"ColorTransfer":null,"ColorPrimaries":null,"DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Rotation":null,"Comment":null,"TimeBase":"1/1000","CodecTimeBase":null,"Title":"English / Forced / ASS","VideoRange":0,"VideoRangeType":0,"VideoDoViTitle":null,"AudioSpatialFormat":0,"LocalizedUndefined":"Undefined","LocalizedDefault":"Default","LocalizedForced":"Forced","LocalizedExternal":"External","LocalizedHearingImpaired":"Hearing Impaired","DisplayTitle":"English / Forced / ASS","NalLengthSize":null,"IsInterlaced":false,"IsAVC":false,"ChannelLayout":null,"BitRate":null,"BitDepth":null,"RefFrames":null,"PacketLength":null,"Channels":null,"SampleRate":null,"IsDefault":false,"IsForced":true,"IsHearingImpaired":false,"Height":0,"Width":0,"AverageFrameRate":null,"RealFrameRate":null,"ReferenceFrameRate":null,"Profile":null,"Type":2,"AspectRatio":null,"Index":4,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":true,"SupportsExternalStream":true,"Path":null,"PixelFormat":null,"Level":0,"IsAnamorphic":null},{"Codec":"ass","CodecTag":null,"Language":"eng","ColorRange":null,"ColorSpace":null,"ColorTransfer":null,"ColorPrimaries":null,"DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Rotation":null,"Comment":null,"TimeBase":"1/1000","CodecTimeBase":null,"Title":"English / Full / ASS","VideoRange":0,"VideoRangeType":0,"VideoDoViTitle":null,"AudioSpatialFormat":0,"LocalizedUndefined":"Undefined","LocalizedDefault":"Default","LocalizedForced":"Forced","LocalizedExternal":"External","LocalizedHearingImpaired":"Hearing Impaired","DisplayTitle":"English / Full / ASS","NalLengthSize":null,"IsInterlaced":false,"IsAVC":false,"ChannelLayout":null,"BitRate":null,"BitDepth":null,"RefFrames":null,"PacketLength":null,"Channels":null,"SampleRate":null,"IsDefault":true,"IsForced":false,"IsHearingImpaired":false,"Height":0,"Width":0,"AverageFrameRate":null,"RealFrameRate":null,"ReferenceFrameRate":null,"Profile":null,"Type":2,"AspectRatio":null,"Index":5,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":true,"SupportsExternalStream":true,"Path":null,"PixelFormat":null,"Level":0,"IsAnamorphic":null},{"Codec":"mjpeg","CodecTag":null,"Language":null,"ColorRange":null,"ColorSpace":"bt470bg","ColorTransfer":null,"ColorPrimaries":null,"DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Rotation":null,"Comment":null,"TimeBase":"1/90000","CodecTimeBase":null,"Title":null,"VideoRange":0,"VideoRangeType":0,"VideoDoViTitle":null,"AudioSpatialFormat":0,"LocalizedUndefined":null,"LocalizedDefault":null,"LocalizedForced":null,"LocalizedExternal":null,"LocalizedHearingImpaired":null,"DisplayTitle":null,"NalLengthSize":null,"IsInterlaced":false,"IsAVC":false,"ChannelLayout":null,"BitRate":null,"BitDepth":8,"RefFrames":1,"PacketLength":null,"Channels":null,"SampleRate":null,"IsDefault":false,"IsForced":false,"IsHearingImpaired":false,"Height":566,"Width":400,"AverageFrameRate":null,"RealFrameRate":90000,"ReferenceFrameRate":90000,"Profile":"Progressive","Type":3,"AspectRatio":"200:283","Index":6,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Path":null,"PixelFormat":"yuvj444p","Level":-99,"IsAnamorphic":false}],"MediaAttachments":[],"Formats":[],"Bitrate":44977834,"FallbackMaxStreamingBitrate":null,"Timestamp":null,"RequiredHttpHeaders":{},"TranscodingUrl":null,"TranscodingSubProtocol":0,"TranscodingContainer":null,"AnalyzeDurationMs":null,"DefaultAudioStreamIndex":null,"DefaultSubtitleStreamIndex":null,"HasSegments":false}
/usr/lib/jellyfin-ffmpeg/ffmpeg -analyzeduration 200M -probesize 1G -fflags +genpts -f matroska -i file:"/data/Media/Video#1.mkv" -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:2 -map -0:s -codec:v:0 copy -bsf:v h264_mp4toannexb -start_at_zero -codec:a:0 copy -copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 -f hls -max_delay 5000000 -hls_time 1 -hls_segment_type fmp4 -hls_fmp4_init_filename "da62577decc74f6260d474e077693870-1.mp4" -start_number 0 -hls_segment_filename "/var/cache/jellyfin/transcodes/da62577decc74f6260d474e077693870%d.mp4" -hls_playlist_type vod -hls_list_size 0 -y "/var/cache/jellyfin/transcodes/da62577decc74f6260d474e077693870.m3u8"