Haswell Hardware Transcoding?

[JdeFalconr]

Thanks in advance for your help.

I'm hoping for some help getting some form of hardware transcoding working with my antiquated E3-1265Lv3; anything will be better than the alternative. From what I've read elsewhere others have gotten Haswell CPUs to work. This page seems to say that Haswell ought to support a few forms of VAAPI encoding and decoding. While QSV would be very nice - and it's theoretically supported - I'm not trying for it at this time.

My platform is Ubuntu Server 24.04.2 LTS in a VMWare VM. I've gone through the official install instructions, confirming I see a renderD128 device, and I've made sure to add the jellyfin user to the render user group. v24.35.30872.22 of the intel-opencl-icd is installed (step #5) and I've run the installation in step 6 (sudo apt install -y intel-opencl-icd). I'm wondering if the issue is drivers. Trouble is, I'm not sure which drivers I should install or where to get them.

Anyhow, in the official Jellyfin docs for setting up hardware transcoding on Linux Step 7 is where things break down:

Code:

sudo /usr/lib/jellyfin-ffmpeg/vainfo --display drm --device /dev/dri/renderD128

Trying display: drm
libva info: VA-API version 1.22.0
libva info: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/vmwgfx_drv_video.so
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/vmwgfx_drv_video.so
libva info: Trying to open /usr/lib/dri/vmwgfx_drv_video.so
libva info: Trying to open /usr/local/lib/dri/vmwgfx_drv_video.so
libva info: va_openDriver() returns -1
vaInitialize failed with error code -1 (unknown libva error),exit

Step 8:

Code:

sudo /usr/lib/jellyfin-ffmpeg/ffmpeg -v verbose -init_hw_device vaapi=va:/dev/dri/renderD128 -init_hw_device opencl@va

ffmpeg version 7.1.1-Jellyfin Copyright (c) 2000-2025 the FFmpeg developers
  built with gcc 13 (Ubuntu 13.3.0-6ubuntu2~24.04)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto=auto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-opencl --enable-libdrm --enable-libxml2 --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libharfbuzz --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libsvtav1 --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-vaapi --enable-amf --enable-libvpl --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc
  libavutil      59. 39.100 / 59. 39.100
  libavcodec    61. 19.101 / 61. 19.101
  libavformat    61.  7.100 / 61.  7.100
  libavdevice    61.  3.100 / 61.  3.100
  libavfilter    10.  4.100 / 10.  4.100
  libswscale      8.  3.100 /  8.  3.100
  libswresample  5.  3.100 /  5.  3.100
  libpostproc    58.  3.100 / 58.  3.100
[AVHWDeviceContext @ 0x643188f48740] libva: VA-API version 1.22.0
[AVHWDeviceContext @ 0x643188f48740] libva: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/vmwgfx_drv_video.so
[AVHWDeviceContext @ 0x643188f48740] libva: Trying to open /usr/lib/x86_64-linux-gnu/dri/vmwgfx_drv_video.so
[AVHWDeviceContext @ 0x643188f48740] libva: Trying to open /usr/lib/dri/vmwgfx_drv_video.so
[AVHWDeviceContext @ 0x643188f48740] libva: Trying to open /usr/local/lib/dri/vmwgfx_drv_video.so
[AVHWDeviceContext @ 0x643188f48740] libva: va_openDriver() returns -1
[AVHWDeviceContext @ 0x643188f48740] Failed to initialise VAAPI connection: -1 (unknown libva error).
Device creation failed: -5.
Failed to set value 'vaapi=va:/dev/dri/renderD128' for option 'init_hw_device': Input/output error
Error parsing global options: Input/output error

Probably unsurprisingly videos don't play at all for me; here's a ffmpeg.transcode log from an attempt:

Code:

{"Protocol":0,"Id":"a0e33ea8281c901d7f3fe7ff64487c88","Path":"/var/lib/jellyfin/data/livetv/recordings/Saturday Night Live/Season 50/Saturday Night Live S50E21 Scarlett Johansson; Bad Bunny.ts","EncoderPath":null,"EncoderProtocol":null,"Type":0,"Container":"ts","Size":4753439168,"Name":"Saturday Night Live S50E21 Scarlett Johansson; Bad Bunny","IsRemote":false,"ETag":"15f27c6c9cb9f506fce7ca9d861fef21","RunTimeTicks":56393006110,"ReadAtNativeFramerate":false,"IgnoreDts":false,"IgnoreIndex":false,"GenPtsInput":false,"SupportsTranscoding":true,"SupportsDirectStream":true,"SupportsDirectPlay":true,"IsInfiniteStream":false,"UseMostCompatibleTranscodingProfile":false,"RequiresOpening":false,"OpenToken":null,"RequiresClosing":false,"LiveStreamId":null,"BufferMs":null,"RequiresLooping":false,"SupportsProbing":true,"VideoType":0,"IsoType":null,"Video3DFormat":null,"MediaStreams":[{"Codec":"mpeg2video","CodecTag":null,"Language":null,"ColorRange":null,"ColorSpace":null,"ColorTransfer":null,"ColorPrimaries":null,"DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Rotation":null,"Comment":null,"TimeBase":"1/90000","CodecTimeBase":null,"Title":null,"VideoRange":1,"VideoRangeType":1,"VideoDoViTitle":null,"AudioSpatialFormat":0,"LocalizedUndefined":null,"LocalizedDefault":null,"LocalizedForced":null,"LocalizedExternal":null,"LocalizedHearingImpaired":null,"DisplayTitle":"1080i MPEG2VIDEO SDR","NalLengthSize":null,"IsInterlaced":true,"IsAVC":false,"ChannelLayout":null,"BitRate":6743303,"BitDepth":8,"RefFrames":1,"PacketLength":null,"Channels":null,"SampleRate":null,"IsDefault":false,"IsForced":false,"IsHearingImpaired":false,"Height":1080,"Width":1920,"AverageFrameRate":29.97003,"RealFrameRate":59.94006,"ReferenceFrameRate":29.97003,"Profile":"Main","Type":1,"AspectRatio":"16:9","Index":0,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Path":null,"PixelFormat":"yuv420p","Level":4,"IsAnamorphic":false},{"Codec":"ac3","CodecTag":"AC-3","Language":"eng","ColorRange":null,"ColorSpace":null,"ColorTransfer":null,"ColorPrimaries":null,"DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Rotation":null,"Comment":null,"TimeBase":"1/90000","CodecTimeBase":null,"Title":null,"VideoRange":0,"VideoRangeType":0,"VideoDoViTitle":null,"AudioSpatialFormat":0,"LocalizedUndefined":null,"LocalizedDefault":"Default","LocalizedForced":null,"LocalizedExternal":"External","LocalizedHearingImpaired":null,"DisplayTitle":"English - Dolby Digital - 5.1","NalLengthSize":null,"IsInterlaced":false,"IsAVC":false,"ChannelLayout":"5.1","BitRate":384000,"BitDepth":null,"RefFrames":null,"PacketLength":null,"Channels":6,"SampleRate":48000,"IsDefault":false,"IsForced":false,"IsHearingImpaired":false,"Height":null,"Width":null,"AverageFrameRate":null,"RealFrameRate":null,"ReferenceFrameRate":null,"Profile":null,"Type":0,"AspectRatio":null,"Index":1,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Path":null,"PixelFormat":null,"Level":0,"IsAnamorphic":null},{"Codec":"ac3","CodecTag":"AC-3","Language":"spa","ColorRange":null,"ColorSpace":null,"ColorTransfer":null,"ColorPrimaries":null,"DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Rotation":null,"Comment":null,"TimeBase":"1/90000","CodecTimeBase":null,"Title":null,"VideoRange":0,"VideoRangeType":0,"VideoDoViTitle":null,"AudioSpatialFormat":0,"LocalizedUndefined":null,"LocalizedDefault":"Default","LocalizedForced":null,"LocalizedExternal":"External","LocalizedHearingImpaired":null,"DisplayTitle":"Spanish - Dolby Digital - Stereo","NalLengthSize":null,"IsInterlaced":false,"IsAVC":false,"ChannelLayout":"stereo","BitRate":128000,"BitDepth":null,"RefFrames":null,"PacketLength":null,"Channels":2,"SampleRate":48000,"IsDefault":false,"IsForced":false,"IsHearingImpaired":false,"Height":null,"Width":null,"AverageFrameRate":null,"RealFrameRate":null,"ReferenceFrameRate":null,"Profile":null,"Type":0,"AspectRatio":null,"Index":2,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Path":null,"PixelFormat":null,"Level":0,"IsAnamorphic":null}],"MediaAttachments":[],"Formats":[],"Bitrate":7255303,"FallbackMaxStreamingBitrate":null,"Timestamp":0,"RequiredHttpHeaders":{},"TranscodingUrl":null,"TranscodingSubProtocol":0,"TranscodingContainer":null,"AnalyzeDurationMs":null,"DefaultAudioStreamIndex":null,"DefaultSubtitleStreamIndex":null,"HasSegments":false}

/usr/lib/jellyfin-ffmpeg/ffmpeg -analyzeduration 200M -probesize 1G -f mpegts -hwaccel vaapi -hwaccel_output_format vaapi -noautorotate -i file:"/var/lib/jellyfin/data/livetv/recordings/Saturday Night Live/Season 50/Saturday Night Live S50E21 Scarlett Johansson; Bad Bunny.ts" -noautoscale -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:1 -map -0:s -codec:v:0 av1_vaapi -rc_mode VBR -b:v 6743303 -maxrate 6743303 -bufsize 13486606 -profile:v:0 main -force_key_frames:0 "expr:gte(t,n_forced*3)" -vf "setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,deinterlace_vaapi=rate=frame,scale_vaapi=format=nv12:extra_hw_frames=24" -codec:a:0 libfdk_aac -ac 6 -ab 640000 -copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 -f hls -max_delay 5000000 -hls_time 3 -hls_segment_type fmp4 -hls_fmp4_init_filename "8c8cb4b2f96a372e2fc1f9e5cc6954f9-1.mp4" -start_number 0 -hls_segment_filename "/var/cache/jellyfin/transcodes/8c8cb4b2f96a372e2fc1f9e5cc6954f9%d.mp4" -hls_playlist_type vod -hls_list_size 0 -y "/var/cache/jellyfin/transcodes/8c8cb4b2f96a372e2fc1f9e5cc6954f9.m3u8"


ffmpeg version 7.1.1-Jellyfin Copyright (c) 2000-2025 the FFmpeg developers
  built with gcc 13 (Ubuntu 13.3.0-6ubuntu2~24.04)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto=auto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-opencl --enable-libdrm --enable-libxml2 --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libharfbuzz --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libsvtav1 --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-vaapi --enable-amf --enable-libvpl --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc
  libavutil      59. 39.100 / 59. 39.100
  libavcodec    61. 19.101 / 61. 19.101
  libavformat    61.  7.100 / 61.  7.100
  libavdevice    61.  3.100 / 61.  3.100
  libavfilter    10.  4.100 / 10.  4.100
  libswscale      8.  3.100 /  8.  3.100
  libswresample  5.  3.100 /  5.  3.100
  libpostproc    58.  3.100 / 58.  3.100
[mpeg2video @ 0x62e8dae61c40] Invalid frame dimensions 0x0.
    Last message repeated 15 times
[mpegts @ 0x62e8dae41f00] PES packet size mismatch
[mpegts @ 0x62e8dae41f00] Packet corrupt (stream = 1, dts = 3151638590).
[mpegts @ 0x62e8dae41f00] PES packet size mismatch
[mpegts @ 0x62e8dae41f00] Packet corrupt (stream = 2, dts = 3151655870).
Input #0, mpegts, from 'file:/var/lib/jellyfin/data/livetv/recordings/Saturday Night Live/Season 50/Saturday Night Live S50E21 Scarlett Johansson; Bad Bunny.ts':
  Duration: 01:33:59.30, start: 29379.742556, bitrate: 6743 kb/s
  Program 3
  Stream #0:0[0x31]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv, top first), 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 59.94 tbr, 90k tbn
      Side data:
        cpb: bitrate max/min/avg: 19400000/0/0 buffer size: 7995392 vbv_delay: N/A
  Stream #0:1[0x34](eng): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 5.1(side), fltp, 384 kb/s
  Stream #0:2[0x35](spa): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, stereo, fltp, 128 kb/s
[AVHWDeviceContext @ 0x62e8dae75c00] Failed to initialise VAAPI connection: -1 (unknown libva error).
Device creation failed: -5.
[vist#0:0/mpeg2video @ 0x62e8dae91dc0] [dec:mpeg2video @ 0x62e8dae91f40] No device available for decoder: device type vaapi needed for codec mpeg2video.
[vist#0:0/mpeg2video @ 0x62e8dae91dc0] [dec:mpeg2video @ 0x62e8dae91f40] Hardware device setup failed for decoder: Input/output error
Error opening output file /var/cache/jellyfin/transcodes/8c8cb4b2f96a372e2fc1f9e5cc6954f9.m3u8.
Error opening output files: Input/output error

[nyanmisaka]

VAAPI cannot access iGPU video codecs in VMware virtual machines. A bare metal Linux installation is required.