SOLVED: Error on trying to transcode hevc (native) -> h264 (h264_nvenc)

[whiskers]

Debian 12
Jellyfin.Server 10.8.13.0

When I try to play an h264 file using the web client, I see this error in the transcoding logs:

Code:

/usr/lib/jellyfin-ffmpeg/ffmpeg -analyzeduration 200M -init_hw_device cuda=cu:0 -filter_hw_device cu -hwaccel cuda -hwaccel_output_format cuda -threads 1 -autorotate 0 -i file:"<video file location>.mkv" -autoscale 0 -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:1 -map -0:s -codec:v:0 h264_nvenc -preset p1 -b:v 8616627 -maxrate 8616627 -bufsize 17233254 -g:v:0 75 -keyint_min:v:0 75 -vf "setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale_cuda=format=yuv420p" -codec:a:0 libfdk_aac -ac 2 -ab 384000 -af "volume=2" -copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 -f hls -max_delay 5000000 -hls_time 3 -hls_segment_type mpegts -start_number 0 -hls_segment_filename "/var/lib/jellyfin/transcodes/f699995509501a55813e79e2da5915e3%d.ts" -hls_playlist_type vod -hls_list_size 0 -y "/var/lib/jellyfin/transcodes/f699995509501a55813e79e2da5915e3.m3u8"


ffmpeg version 5.1.4-Jellyfin Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-libs=-lfftw3f --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libsvtav1 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-opencl --enable-vaapi --enable-amf --enable-libmfx --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc
  libavutil      57. 28.100 / 57. 28.100
  libavcodec    59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter    8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample  4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
Input #0, matroska,webm, from 'file:<video file location>.mkv':
  Metadata:
    creation_time  : 2023-07-11T17:19:43.000000Z
    ENCODER        : Lavf59.27.100
  Duration: 00:25:11.55, start: 0.000000, bitrate: 2584 kb/s
  Stream #0:0: Video: hevc (Main), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 1k tbn (default)
    Metadata:
      DURATION        : 00:25:11.520000000
  Stream #0:1(eng): Audio: eac3, 48000 Hz, 5.1(side), fltp, 640 kb/s (default)
    Metadata:
      title          : Surround
      DURATION        : 00:25:11.552000000
  Stream #0:2(eng): Subtitle: ass
    Metadata:
      DURATION        : 00:25:05.880000000
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> h264 (h264_nvenc))
  Stream #0:1 -> #0:1 (eac3 (native) -> aac (libfdk_aac))
Press [q] to stop, [?] for help
[hevc @ 0x55b2cd84ebc0] Hardware is lacking required capabilities
[hevc @ 0x55b2cd84ebc0] Failed setup for format cuda: hwaccel initialisation returned error.
Impossible to convert between the formats supported by the filter 'Parsed_setparams_0' and the filter 'auto_scale_0'
Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented
Error while processing the decoded data for stream #0:0
Conversion failed!

I can reproduce the error by running the ffmpeg command in my terminal. When I google the error I see: https://stackoverflow.com/questions/5202...lter-error

which says I need to pass in "hwupload_cuda" along with the filter. When I add it I get the following output in my terminal:

Code:

  Stream #0:0 -> #0:0 (hevc (native) -> h264 (h264_nvenc))
  Stream #0:1 -> #0:1 (ac3 (native) -> aac (libfdk_aac))
Press [q] to stop, [?] for help
[hevc @ 0x558a1a387140] Hardware is lacking required capabilities
[hevc @ 0x558a1a387140] Failed setup for format cuda: hwaccel initialisation returned error.
Output #0, hls, to '/var/lib/jellyfin/transcodes/17077ca8b9dbdc387cfcecd71dd32fb8.m3u8':
  Metadata:
    encoder        : Lavf59.27.100
  Stream #0:0: Video: h264 (Main), cuda(tv, bt709, progressive), 1920x800 [SAR 1:1 DAR 12:5], q=2-31, 9485 kb/s, 23.98 fps, 90k tbn (default)
    Metadata:
      encoder        : Lavc59.37.100 h264_nvenc
    Side data:
      cpb: bitrate max/min/avg: 9485480/0/9485480 buffer size: 18970960 vbv_delay: N/A
  Stream #0:1: Audio: aac, 48000 Hz, stereo, s16, 384 kb/s (default)
    Metadata:
      encoder        : Lavc59.37.100 libfdk_aac
[hls @ 0x558a1a379ec0] Opening '/var/lib/jellyfin/transcodes/17077ca8b9dbdc387cfcecd71dd32fb80.ts' for writing
[hls @ 0x558a1a379ec0] Opening '/var/lib/jellyfin/transcodes/17077ca8b9dbdc387cfcecd71dd32fb81.ts' for writing
[hls @ 0x558a1a379ec0] Opening '/var/lib/jellyfin/transcodes/17077ca8b9dbdc387cfcecd71dd32fb82.ts' for writing
[hls @ 0x558a1a379ec0] Opening '/var/lib/jellyfin/transcodes/17077ca8b9dbdc387cfcecd71dd32fb83.ts' for writing
[hls @ 0x558a1a379ec0] Opening '/var/lib/jellyfin/transcodes/17077ca8b9dbdc387cfcecd71dd32fb84.ts' for writing
[hls @ 0x558a1a379ec0] Opening '/var/lib/jellyfin/transcodes/17077ca8b9dbdc387cfcecd71dd32fb85.ts' for writing
[hls @ 0x558a1a379ec0] Opening '/var/lib/jellyfin/transcodes/17077ca8b9dbdc387cfcecd71dd32fb86.ts' for writing
[hls @ 0x558a1a379ec0] Opening '/var/lib/jellyfin/transcodes/17077ca8b9dbdc387cfcecd71dd32fb87.ts' for writing
[hls @ 0x558a1a379ec0] Opening '/var/lib/jellyfin/transcodes/17077ca8b9dbdc387cfcecd71dd32fb88.ts' for writing
[hls @ 0x558a1a379ec0] Opening '/var/lib/jellyfin/transcodes/17077ca8b9dbdc387cfcecd71dd32fb89.ts' for writing
frame=  708 fps=112 q=19.0 Lsize=N/A time=00:00:25.66 bitrate=N/A speed=4.05x
video:29823kB audio:1383kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Exiting normally, received signal 2.

So (best I can tell) it says hwaccel initialization error but then goes ahead and does the decode anyway?

Output of lspci for gpu:

Code:

root@:/# lspci -nn | grep -Ei "3d|display|vga"
01:00.1 VGA compatible controller [0300]: Matrox Electronics Systems Ltd. MGA G200eH3 [] (rev 02)
af:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM200 [GeForce GTX TITAN X] [] (rev a1)

Output of nvidia-smi:

Code:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.147.05  Driver Version: 525.147.05  CUDA Version: 12.0    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|        Memory-Usage | GPU-Util  Compute M. |
|                              |                      |              MIG M. |
|===============================+======================+======================|
|  0  NVIDIA GeForce ...  Off  | 00000000:AF:00.0 Off |                  N/A |
| 18%  57C    P0    64W / 250W |      0MiB / 12288MiB |      1%      Default |
|                              |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU  GI  CI        PID  Type  Process name                  GPU Memory |
|        ID  ID                                                  Usage      |
|=============================================================================|
|  No running processes found                                                |
+-----------------------------------------------------------------------------+

Anyone have ideas on how to resolve this?