Login

grips · 2024-04-16, 04:50 PM

Hello,

I am having some transcoding issues with certain files, and it seems to be related to hardware encoding with Apple VideoToolBox.

I have Jellyfin installed on an Intel-based Mac Mini using the v10.8.13 dmg.

The ffmpeg log looks like:

Quote:ffmpeg -analyzeduration 200M -init_hw_device videotoolbox=vt -hwaccel videotoolbox -autorotate 0 -i file:"/Volumes/video/Series/MacGyver/Season 1/01 Pilot-C.avi" -autoscale 0 -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:1 -map -0:s -codec:v:0 h264_videotoolbox -b:v 3203195 -maxrate 3203195 -bufsize 6406390 -force_key_frames:0 "expr:gte(t,0+n_forced*3)" -g:v:0 72 -keyint_min:v:0 72 -vf "setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale=trunc(min(max(iw\,ih*a)\,min(512\,384*a))/2)*2:trunc(min(max(iw/a\,ih)\,min(512/a\,384))/2)*2,format=nv12" -codec:a:0 aac -ac 2 -ab 256000 -copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 -f hls -max_delay 5000000 -hls_time 3 -hls_segment_type mpegts -start_number 0 -hls_segment_filename "/Users/xxx/.local/share/jellyfin/transcodes/7736832436128392456bd5ce87ea24d7%d.ts" -hls_playlist_type vod -hls_list_size 0 -y "/Users/xxx/.local/share/jellyfin/transcodes/7736832436128392456bd5ce87ea24d7.m3u8"

ffmpeg version 5.1.2-tessus Copyright © 2000-2022 the FFmpeg developers
built with Apple clang version 11.0.0 (clang-1100.0.33.17)
configuration: --cc=/usr/bin/clang --prefix=/opt/ffmpeg --extra-version=tessus --enable-avisynth --enable-fontconfig --enable-gpl --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libfreetype --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libmysofa --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvmaf --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-version3 --pkg-config-flags=--static --disable-ffplay
libavutil 57. 28.100 / 57. 28.100
libavcodec 59. 37.100 / 59. 37.100
libavformat 59. 27.100 / 59. 27.100
libavdevice 59. 7.100 / 59. 7.100
libavfilter 8. 44.100 / 8. 44.100
libswscale 6. 7.100 / 6. 7.100
libswresample 4. 7.100 / 4. 7.100
libpostproc 56. 6.100 / 56. 6.100
Input #0, avi, from 'file:/Volumes/video/Series/MacGyver/Season 1/01 Pilot-C.avi':
Metadata:
software : Lavf52.32.0
Duration: 00:45:40.62, start: 0.000000, bitrate: 618 kb/s
Stream #0:0: Video: mpeg4 (Simple Profile) (xvid / 0x64697678), yuv420p, 512x384 [SAR 1:1 DAR 4:3], 512 kb/s, 23.98 fps, 23.98 tbr, 23.98 tbn
Stream #0:1: Audio: ac3 ([0] [0][0] / 0x2000), 44100 Hz, stereo, fltp, 96 kb/s
Stream mapping:
Stream #0:0 -> #0:0 (mpeg4 (native) -> h264 (h264_videotoolbox))
Stream #0:1 -> #0:1 (ac3 (native) -> aac (native))
Press [q] to stop, [?] for help
[mpeg4 @ 0x7f80a600a980] Failed setup for format videotoolbox_vld: hwaccel initialisation returned error.
[h264_videotoolbox @ 0x7f80a60085c0] Error: cannot create compression session: -12903
[h264_videotoolbox @ 0x7f80a60085c0] Try -allow_sw 1. The hardware encoder may be busy, or not supported.
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
[aac @ 0x7f80a6009840] Qavg: 623.956
[aac @ 0x7f80a6009840] 2 frames left in the queue on closing
Conversion failed!

It does work properly with other video files. I'm looking for ideas on resolving this. Let me know if I can provide any extra details to help.

Thank you

TheDreadPirate · 2024-04-16, 05:00 PM

What model CPU? And can you share a screenshot of the codec boxes in Dashboard > Playback?

grips

Quote: Model Name: Mac mini
Model Identifier: Macmini8,1
Processor Name: 6-Core Intel Core i7
Processor Speed: 3.2 GHz
Number of Processors: 1
Total Number of Cores: 6
L2 Cache (per Core): 256 KB
L3 Cache: 12 MB

Is this enough detail on the CPU?

Edit: looks to be a Core i7 (I7-8700B)

[Image: Screenshot-2024-04-16-at-12-08-12-PM.png]

TheDreadPirate · 2024-04-16, 05:20 PM

Core i7 is not enough. I need the specific model like "Core i3 12100". Based on what I can find for "macmini8,1" it looks like an 8th generation Intel CPU? Is that correct?

If so, uncheck MPEG4 and save. AFAIK, Intel iGPUs cannot decode that codec and your video uses MPEG4. Unchecking the box just means the CPU will decode the video.

**gnattu**

Your ffmpeg is not configured with videotoolbox enabled, and current 10.8.13 dmg has a bug that cannot use the correct ffmpeg that comes with jellyfin.

A workaround is to manually download the jellyfin's ffmpeg from the repo: https://repo.jellyfin.org/?path=/ffmpeg/...6.x/x86_64

and then manually edit the encoding.xml to let jellyfin find that ffmpeg.

Alternatively you can use jellyfin's ffmpeg to replace your system's ffmpeg.

You do not have to do these in the following 10.9 release.

grips · 2024-04-16, 05:44 PM

(2024-04-16, 05:30 PM)gnattu Wrote: Your ffmpeg is not configured with videotoolbox enabled, and current 10.8.13 dmg has a bug that cannot use the correct ffmpeg that comes with jellyfin.

A workaround is to manually download the jellyfin's ffmpeg from the repo: https://repo.jellyfin.org/?path=/ffmpeg/...6.x/x86_64

and then manually edit the encoding.xml to let jellyfin find that ffmpeg.

Alternatively you can use jellyfin's ffmpeg to replace your system's ffmpeg.

You do not have to do these in the following 10.9 release.

These errors are generated with jellfyin-ffmpeg.

Code:
xxx@Singularity bin % ll|grep ffmpeg

lrwxr-xr-x    1 xxx  admin    73 Apr 16 11:54 ffmpeg -> /Users/xxx/Downloads/jellyfin-ffmpeg_6.0.1-5_portable_mac64-gpl/ffmpeg

lrwxr-xr-x    1 xxx  admin    74 Apr 16 11:54 ffprobe -> /Users/xxx/Downloads/jellyfin-ffmpeg_6.0.1-5_portable_mac64-gpl/ffprobe

xxx@Singularity bin % pwd

/usr/local/bin

xxx@Singularity bin %

Code:
  <HardwareAccelerationType>videotoolbox</HardwareAccelerationType>

  <EncoderAppPath>/usr/local/bin/ffmpeg</EncoderAppPath>

  <EncoderAppPathDisplay>/usr/local/bin/ffmpeg</EncoderAppPathDisplay>

Quote:Core i7 is not enough. I need the specific model like "Core i3 12100". Based on what I can find for "macmini8,1" it looks like an 8th generation Intel CPU? Is that correct?

If so, uncheck MPEG4 and save. AFAIK, Intel iGPUs cannot decode that codec and your video uses MPEG4. Unchecking the box just means the CPU will decode the video.

Sorry, I edited the earlier post with the details instead of responding. It seems to be a Core i7 (I7-8700B). I tried unchecking the MPEG4 and it still failed, but the message is actually slightly different.

Quote:Input #0, avi, from 'fileVolumes/video/Series/MacGyver/Season 1/01 Pilot-C.avi':
Metadata:
software : Lavf52.32.0
Duration: 00:45:40.62, start: 0.000000, bitrate: 618 kb/s
Stream #0:0: Video: mpeg4 (Simple Profile) (xvid / 0x6469767, yuv420p, 512x384 [SAR 1:1 DAR 4:3], 512 kb/s, 23.98 fps, 23.98 tbr, 23.98 tbn
Stream #0:1: Audio: ac3 ([0] [0][0] / 0x2000), 44100 Hz, stereo, fltp, 96 kb/s
Stream mapping:
Stream #0:0 -> #0:0 (mpeg4 (native) -> h264 (h264_videotoolbox))
Stream #0:1 -> #0:1 (ac3 (native) -> aac (native))
Press [q] to stop, [?] for help
[h264_videotoolbox @ 0x7fd3a6908d80] Error: cannot create compression session: -12903
[h264_videotoolbox @ 0x7fd3a6908d80] Try -allow_sw 1. The hardware encoder may be busy, or not supported.
[vost#0:0/h264_videotoolbox @ 0x7fd3a6908a80] Error initializing output stream: Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
[aac @ 0x7fd3a6909ec0] Qavg: 921.578
[aac @ 0x7fd3a6909ec0] 2 frames left in the queue on closing
Conversion failed!

**gnattu** · 2024-04-16, 06:05 PM

(2024-04-16, 05:44 PM)grips Wrote: These errors are generated with jellfyin-ffmpeg.

Code:
xxx@Singularity bin % ll|grep ffmpeg lrwxr-xr-x 1 xxx admin 73 Apr 16 11:54 ffmpeg -> /Users/xxx/Downloads/jellyfin-ffmpeg_6.0.1-5_portable_mac64-gpl/ffmpeg lrwxr-xr-x 1 xxx admin 74 Apr 16 11:54 ffprobe -> /Users/xxx/Downloads/jellyfin-ffmpeg_6.0.1-5_portable_mac64-gpl/ffprobe xxx@Singularity bin % pwd /usr/local/bin xxx@Singularity bin %

Code:
<HardwareAccelerationType>videotoolbox</HardwareAccelerationType> <EncoderAppPath>/usr/local/bin/ffmpeg</EncoderAppPath> <EncoderAppPathDisplay>/usr/local/bin/ffmpeg</EncoderAppPathDisplay>

Quote:Input #0, avi, from 'fileVolumes/video/Series/MacGyver/Season 1/01 Pilot-C.avi':
Metadata:
software : Lavf52.32.0
Duration: 00:45:40.62, start: 0.000000, bitrate: 618 kb/s
Stream #0:0: Video: mpeg4 (Simple Profile) (xvid / 0x6469767, yuv420p, 512x384 [SAR 1:1 DAR 4:3], 512 kb/s, 23.98 fps, 23.98 tbr, 23.98 tbn
Stream #0:1: Audio: ac3 ([0] [0][0] / 0x2000), 44100 Hz, stereo, fltp, 96 kb/s
Stream mapping:
Stream #0:0 -> #0:0 (mpeg4 (native) -> h264 (h264_videotoolbox))
Stream #0:1 -> #0:1 (ac3 (native) -> aac (native))
Press [q] to stop, [?] for help
[h264_videotoolbox @ 0x7fd3a6908d80] Error: cannot create compression session: -12903
[h264_videotoolbox @ 0x7fd3a6908d80] Try -allow_sw 1. The hardware encoder may be busy, or not supported.
[vost#0:0/h264_videotoolbox @ 0x7fd3a6908a80] Error initializing output stream: Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
[aac @ 0x7fd3a6909ec0] Qavg: 921.578
[aac @ 0x7fd3a6909ec0] 2 frames left in the queue on closing
Conversion failed!

I need to see the *full log*. Please please please don't cut off the log as the part that you *think* is relevant may not be helpful.

grips · 2024-04-16, 06:17 PM

Sure thing. Here is the full FFmpeg log with MPEG4 unchecked. Let me know if anything else would help.

Quote:/videos/52f05471-6b03-6882-790c-f8bcebbe8c1b/hls1/main/0.ts

{"Protocol":0,"Id":"52f054716b036882790cf8bcebbe8c1b","Path":"/Volumes/video/Series/MacGyver/Season 1/01 Pilot-C.avi","EncoderPath":null,"EncoderProtocol":null,"Type":0,"Container":"avi","Size":211960960,"Name":"01 Pilot-C","IsRemote":false,"ETag":"77d983fec07f6590e5fe5dc0067f0c52","RunTimeTicks":27406155776,"ReadAtNativeFramerate":false,"IgnoreDts":false,"IgnoreIndex":false,"GenPtsInput":false,"SupportsTranscoding":true,"SupportsDirectStream":true,"SupportsDirectPlay":true,"IsInfiniteStream":false,"RequiresOpening":false,"OpenToken":null,"RequiresClosing":false,"LiveStreamId":null,"BufferMs":null,"RequiresLooping":false,"SupportsProbing":true,"VideoType":0,"IsoType":null,"Video3DFormat":null,"MediaStreams":[{"Codec":"mpeg4","CodecTag":"xvid","Language":null,"ColorRange":null,"ColorSpace":null,"ColorTransfer":null,"ColorPrimaries":null,"DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Comment":null,"TimeBase":"125/2997","CodecTimeBase":null,"Title":null,"VideoRange":"SDR","VideoRangeType":"SDR","VideoDoViTitle":null,"LocalizedUndefined":null,"LocalizedDefault":null,"LocalizedForced":null,"LocalizedExternal":null,"DisplayTitle":"384p MPEG4 SDR","NalLengthSize":null,"IsInterlaced":false,"IsAVC":null,"ChannelLayout":null,"BitRate":512511,"BitDepth":8,"RefFrames":1,"PacketLength":null,"Channels":null,"SampleRate":null,"IsDefault":false,"IsForced":false,"Height":384,"Width":512,"AverageFrameRate":23.976,"RealFrameRate":23.976,"Profile":"Simple Profile","Type":1,"AspectRatio":"4:3","Index":0,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Path":null,"PixelFormat":"yuv420p","Level":3,"IsAnamorphic":null},{"Codec":"ac3","CodecTag":null,"Language":null,"ColorRange":null,"ColorSpace":null,"ColorTransfer":null,"ColorPrimaries":null,"DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Comment":null,"TimeBase":"128/3675","CodecTimeBase":null,"Title":null,"VideoRange":null,"VideoRangeType":null,"VideoDoViTitle":null,"LocalizedUndefined":null,"LocalizedDefault":null,"LocalizedForced":null,"LocalizedExternal":null,"DisplayTitle":"Dolby Digital - Stereo","NalLengthSize":null,"IsInterlaced":false,"IsAVC":null,"ChannelLayout":"stereo","BitRate":96000,"BitDepth":null,"RefFrames":null,"PacketLength":null,"Channels":2,"SampleRate":44100,"IsDefault":false,"IsForced":false,"Height":null,"Width":null,"AverageFrameRate":null,"RealFrameRate":null,"Profile":null,"Type":0,"AspectRatio":null,"Index":1,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Path":null,"PixelFormat":null,"Level":0,"IsAnamorphic":null}],"MediaAttachments":[],"Formats":[],"Bitrate":618725,"Timestamp":null,"RequiredHttpHeaders":{},"TranscodingUrl":null,"TranscodingSubProtocol":null,"TranscodingContainer":null,"AnalyzeDurationMs":null,"DefaultAudioStreamIndex":null,"DefaultSubtitleStreamIndex":null}

/usr/local/bin/ffmpeg -analyzeduration 200M -init_hw_device videotoolbox=vt -autorotate 0 -i file:"/Volumes/video/Series/MacGyver/Season 1/01 Pilot-C.avi" -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:1 -map -0:s -codec:v:0 h264_videotoolbox -b:v 3203195 -maxrate 3203195 -bufsize 6406390 -force_key_frames:0 "expr:gte(t,0+n_forced*3)" -g:v:0 72 -keyint_min:v:0 72 -vf "setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale=trunc(min(max(iw\,ih*a)\,min(512\,384*a))/2)*2:trunc(min(max(iw/a\,ih)\,min(512/a\,384))/2)*2,format=yuv420p" -codec:a:0 aac -ac 2 -ab 256000 -copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 -f hls -max_delay 5000000 -hls_time 3 -hls_segment_type mpegts -start_number 0 -hls_segment_filename "/Users/xxx/.local/share/jellyfin/transcodes/11f680a00e47b9fe31944789a117e0aa%d.ts" -hls_playlist_type vod -hls_list_size 0 -y "/Users/xxx/.local/share/jellyfin/transcodes/11f680a00e47b9fe31944789a117e0aa.m3u8"

ffmpeg version 6.0.1-Jellyfin Copyright © 2000-2023 the FFmpeg developers
built with Apple clang version 14.0.0 (clang-1400.0.29.202)
configuration: --prefix=/ffbuild/prefix --disable-shared --enable-static --pkg-config-flags=--static --enable-pthreads --cc=clang --host-cflags='-I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/opt/ffbuild/prefix/include' --host-ldflags='-L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib -L/opt/ffbuild/prefix/lib' --extra-version=Jellyfin --extra-cflags=-I/opt/ffbuild/prefix/include --extra-cxxflags= --extra-ldflags=-L/opt/ffbuild/prefix/lib --extra-ldexeflags= --extra-libs= --enable-gpl --enable-version3 --enable-lto --disable-ffplay --disable-debug --disable-doc --disable-ptx-compression --disable-sdl2 --disable-libjack --disable-indev=jack --enable-neon --enable-runtime-cpudetect --enable-audiotoolbox --enable-videotoolbox --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-openssl --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --disable-amf --enable-chromaprint --enable-libdav1d --disable-libfdk-aac --disable-ffnvcodec --disable-cuda --disable-cuda-llvm --disable-cuvid --disable-nvdec --disable-nvenc --enable-libass --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvpx --enable-libwebp --enable-libopenmpt --enable-libsrt --enable-libsvtav1 --enable-libx264 --enable-libx265 --enable-libzimg --enable-libzvbi
libavutil 58. 2.100 / 58. 2.100
libavcodec 60. 3.100 / 60. 3.100
libavformat 60. 3.100 / 60. 3.100
libavdevice 60. 1.100 / 60. 1.100
libavfilter 9. 3.100 / 9. 3.100
libswscale 7. 1.100 / 7. 1.100
libswresample 4. 10.100 / 4. 10.100
libpostproc 57. 1.100 / 57. 1.100
Input #0, avi, from 'fileVolumes/video/Series/MacGyver/Season 1/01 Pilot-C.avi':
Metadata:
software : Lavf52.32.0
Duration: 00:45:40.62, start: 0.000000, bitrate: 618 kb/s
Stream #0:0: Video: mpeg4 (Simple Profile) (xvid / 0x6469767, yuv420p, 512x384 [SAR 1:1 DAR 4:3], 512 kb/s, 23.98 fps, 23.98 tbr, 23.98 tbn
Stream #0:1: Audio: ac3 ([0] [0][0] / 0x2000), 44100 Hz, stereo, fltp, 96 kb/s
Stream mapping:
Stream #0:0 -> #0:0 (mpeg4 (native) -> h264 (h264_videotoolbox))
Stream #0:1 -> #0:1 (ac3 (native) -> aac (native))
Press [q] to stop, [?] for help
[h264_videotoolbox @ 0x7fcb4ff09840] Error: cannot create compression session: -12903
[h264_videotoolbox @ 0x7fcb4ff09840] Try -allow_sw 1. The hardware encoder may be busy, or not supported.
[vost#0:0/h264_videotoolbox @ 0x7fcb4ff09400] Error initializing output stream: Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
[aac @ 0x7fcb4ff0a780] Qavg: 921.578
[aac @ 0x7fcb4ff0a780] 2 frames left in the queue on closing
Conversion failed!

**gnattu** · 2024-04-16, 06:36 PM

This really should not happen and I tried the same command and I cannot reproduce this behavior. What is your macOS version? Just in case it is too old to be supported.

To verify if your system works with hwaccel in general, you can test with:



./ffmpeg -f lavfi -i testsrc=size=512x384:rate=23.98 -t 30 -c:v h264_videotoolbox test.mp4

in the path contains jellyfin-ffmpeg. If it still errors out for this probably minimal use case, then it is not really a jellyfin issue, but rather some problem with:

- macOS on your machine, a not compatible version or a broken installation

- or even worse, a broken Mac (less likely)

grips · 2024-04-16, 06:49 PM

Crazy, that command does fail. I guess it's something with my setup, then. I'm running the latest MacOS.

Other formats seem to work without any issues. I found this old, closed github issue that looks similar: https://github.com/jellyfin/jellyfin/issues/9507

Anyway, I appreciate the time and info. I will try to look into ffmpeg more and see if I can't resolve something.

Login

Username/Email:
Password:


Or login with a social network below

Login

Username/Email:
Password:


Or login with a social network below

Apple VideoToolBox encoding failure