Haswell Hardware Transcoding?

[JdeFalconr]

Thanks in advance for your help.

I'm hoping for some help getting some form of hardware transcoding working with my antiquated E3-1265Lv3; anything will be better than the alternative. From what I've read elsewhere others have gotten Haswell CPUs to work. This page seems to say that Haswell ought to support a few forms of VAAPI encoding and decoding. While QSV would be very nice - and it's theoretically supported - I'm not trying for it at this time.

My platform is Ubuntu Server 24.04.2 LTS in a VMWare VM. I've gone through the official install instructions, confirming I see a renderD128 device, and I've made sure to add the jellyfin user to the render user group. v24.35.30872.22 of the intel-opencl-icd is installed (step #5) and I've run the installation in step 6 (sudo apt install -y intel-opencl-icd). I'm wondering if the issue is drivers. Trouble is, I'm not sure which drivers I should install or where to get them.

Anyhow, in the official Jellyfin docs for setting up hardware transcoding on Linux Step 7 is where things break down:

Code:

sudo /usr/lib/jellyfin-ffmpeg/vainfo --display drm --device /dev/dri/renderD128

Trying display: drm
libva info: VA-API version 1.22.0
libva info: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/vmwgfx_drv_video.so
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/vmwgfx_drv_video.so
libva info: Trying to open /usr/lib/dri/vmwgfx_drv_video.so
libva info: Trying to open /usr/local/lib/dri/vmwgfx_drv_video.so
libva info: va_openDriver() returns -1
vaInitialize failed with error code -1 (unknown libva error),exit

Step 8:

Code:

sudo /usr/lib/jellyfin-ffmpeg/ffmpeg -v verbose -init_hw_device vaapi=va:/dev/dri/renderD128 -init_hw_device opencl@va

ffmpeg version 7.1.1-Jellyfin Copyright (c) 2000-2025 the FFmpeg developers
  built with gcc 13 (Ubuntu 13.3.0-6ubuntu2~24.04)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto=auto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-opencl --enable-libdrm --enable-libxml2 --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libharfbuzz --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libsvtav1 --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-vaapi --enable-amf --enable-libvpl --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc
  libavutil      59. 39.100 / 59. 39.100
  libavcodec    61. 19.101 / 61. 19.101
  libavformat    61.  7.100 / 61.  7.100
  libavdevice    61.  3.100 / 61.  3.100
  libavfilter    10.  4.100 / 10.  4.100
  libswscale      8.  3.100 /  8.  3.100
  libswresample  5.  3.100 /  5.  3.100
  libpostproc    58.  3.100 / 58.  3.100
[AVHWDeviceContext @ 0x643188f48740] libva: VA-API version 1.22.0
[AVHWDeviceContext @ 0x643188f48740] libva: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/vmwgfx_drv_video.so
[AVHWDeviceContext @ 0x643188f48740] libva: Trying to open /usr/lib/x86_64-linux-gnu/dri/vmwgfx_drv_video.so
[AVHWDeviceContext @ 0x643188f48740] libva: Trying to open /usr/lib/dri/vmwgfx_drv_video.so
[AVHWDeviceContext @ 0x643188f48740] libva: Trying to open /usr/local/lib/dri/vmwgfx_drv_video.so
[AVHWDeviceContext @ 0x643188f48740] libva: va_openDriver() returns -1
[AVHWDeviceContext @ 0x643188f48740] Failed to initialise VAAPI connection: -1 (unknown libva error).
Device creation failed: -5.
Failed to set value 'vaapi=va:/dev/dri/renderD128' for option 'init_hw_device': Input/output error
Error parsing global options: Input/output error

Probably unsurprisingly videos don't play at all for me; here's a ffmpeg.transcode log from an attempt:

Code:

{"Protocol":0,"Id":"a0e33ea8281c901d7f3fe7ff64487c88","Path":"/var/lib/jellyfin/data/livetv/recordings/Saturday Night Live/Season 50/Saturday Night Live S50E21 Scarlett Johansson; Bad Bunny.ts","EncoderPath":null,"EncoderProtocol":null,"Type":0,"Container":"ts","Size":4753439168,"Name":"Saturday Night Live S50E21 Scarlett Johansson; Bad Bunny","IsRemote":false,"ETag":"15f27c6c9cb9f506fce7ca9d861fef21","RunTimeTicks":56393006110,"ReadAtNativeFramerate":false,"IgnoreDts":false,"IgnoreIndex":false,"GenPtsInput":false,"SupportsTranscoding":true,"SupportsDirectStream":true,"SupportsDirectPlay":true,"IsInfiniteStream":false,"UseMostCompatibleTranscodingProfile":false,"RequiresOpening":false,"OpenToken":null,"RequiresClosing":false,"LiveStreamId":null,"BufferMs":null,"RequiresLooping":false,"SupportsProbing":true,"VideoType":0,"IsoType":null,"Video3DFormat":null,"MediaStreams":[{"Codec":"mpeg2video","CodecTag":null,"Language":null,"ColorRange":null,"ColorSpace":null,"ColorTransfer":null,"ColorPrimaries":null,"DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Rotation":null,"Comment":null,"TimeBase":"1/90000","CodecTimeBase":null,"Title":null,"VideoRange":1,"VideoRangeType":1,"VideoDoViTitle":null,"AudioSpatialFormat":0,"LocalizedUndefined":null,"LocalizedDefault":null,"LocalizedForced":null,"LocalizedExternal":null,"LocalizedHearingImpaired":null,"DisplayTitle":"1080i MPEG2VIDEO SDR","NalLengthSize":null,"IsInterlaced":true,"IsAVC":false,"ChannelLayout":null,"BitRate":6743303,"BitDepth":8,"RefFrames":1,"PacketLength":null,"Channels":null,"SampleRate":null,"IsDefault":false,"IsForced":false,"IsHearingImpaired":false,"Height":1080,"Width":1920,"AverageFrameRate":29.97003,"RealFrameRate":59.94006,"ReferenceFrameRate":29.97003,"Profile":"Main","Type":1,"AspectRatio":"16:9","Index":0,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Path":null,"PixelFormat":"yuv420p","Level":4,"IsAnamorphic":false},{"Codec":"ac3","CodecTag":"AC-3","Language":"eng","ColorRange":null,"ColorSpace":null,"ColorTransfer":null,"ColorPrimaries":null,"DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Rotation":null,"Comment":null,"TimeBase":"1/90000","CodecTimeBase":null,"Title":null,"VideoRange":0,"VideoRangeType":0,"VideoDoViTitle":null,"AudioSpatialFormat":0,"LocalizedUndefined":null,"LocalizedDefault":"Default","LocalizedForced":null,"LocalizedExternal":"External","LocalizedHearingImpaired":null,"DisplayTitle":"English - Dolby Digital - 5.1","NalLengthSize":null,"IsInterlaced":false,"IsAVC":false,"ChannelLayout":"5.1","BitRate":384000,"BitDepth":null,"RefFrames":null,"PacketLength":null,"Channels":6,"SampleRate":48000,"IsDefault":false,"IsForced":false,"IsHearingImpaired":false,"Height":null,"Width":null,"AverageFrameRate":null,"RealFrameRate":null,"ReferenceFrameRate":null,"Profile":null,"Type":0,"AspectRatio":null,"Index":1,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Path":null,"PixelFormat":null,"Level":0,"IsAnamorphic":null},{"Codec":"ac3","CodecTag":"AC-3","Language":"spa","ColorRange":null,"ColorSpace":null,"ColorTransfer":null,"ColorPrimaries":null,"DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Rotation":null,"Comment":null,"TimeBase":"1/90000","CodecTimeBase":null,"Title":null,"VideoRange":0,"VideoRangeType":0,"VideoDoViTitle":null,"AudioSpatialFormat":0,"LocalizedUndefined":null,"LocalizedDefault":"Default","LocalizedForced":null,"LocalizedExternal":"External","LocalizedHearingImpaired":null,"DisplayTitle":"Spanish - Dolby Digital - Stereo","NalLengthSize":null,"IsInterlaced":false,"IsAVC":false,"ChannelLayout":"stereo","BitRate":128000,"BitDepth":null,"RefFrames":null,"PacketLength":null,"Channels":2,"SampleRate":48000,"IsDefault":false,"IsForced":false,"IsHearingImpaired":false,"Height":null,"Width":null,"AverageFrameRate":null,"RealFrameRate":null,"ReferenceFrameRate":null,"Profile":null,"Type":0,"AspectRatio":null,"Index":2,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Path":null,"PixelFormat":null,"Level":0,"IsAnamorphic":null}],"MediaAttachments":[],"Formats":[],"Bitrate":7255303,"FallbackMaxStreamingBitrate":null,"Timestamp":0,"RequiredHttpHeaders":{},"TranscodingUrl":null,"TranscodingSubProtocol":0,"TranscodingContainer":null,"AnalyzeDurationMs":null,"DefaultAudioStreamIndex":null,"DefaultSubtitleStreamIndex":null,"HasSegments":false}

/usr/lib/jellyfin-ffmpeg/ffmpeg -analyzeduration 200M -probesize 1G -f mpegts -hwaccel vaapi -hwaccel_output_format vaapi -noautorotate -i file:"/var/lib/jellyfin/data/livetv/recordings/Saturday Night Live/Season 50/Saturday Night Live S50E21 Scarlett Johansson; Bad Bunny.ts" -noautoscale -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:1 -map -0:s -codec:v:0 av1_vaapi -rc_mode VBR -b:v 6743303 -maxrate 6743303 -bufsize 13486606 -profile:v:0 main -force_key_frames:0 "expr:gte(t,n_forced*3)" -vf "setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,deinterlace_vaapi=rate=frame,scale_vaapi=format=nv12:extra_hw_frames=24" -codec:a:0 libfdk_aac -ac 6 -ab 640000 -copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 -f hls -max_delay 5000000 -hls_time 3 -hls_segment_type fmp4 -hls_fmp4_init_filename "8c8cb4b2f96a372e2fc1f9e5cc6954f9-1.mp4" -start_number 0 -hls_segment_filename "/var/cache/jellyfin/transcodes/8c8cb4b2f96a372e2fc1f9e5cc6954f9%d.mp4" -hls_playlist_type vod -hls_list_size 0 -y "/var/cache/jellyfin/transcodes/8c8cb4b2f96a372e2fc1f9e5cc6954f9.m3u8"


ffmpeg version 7.1.1-Jellyfin Copyright (c) 2000-2025 the FFmpeg developers
  built with gcc 13 (Ubuntu 13.3.0-6ubuntu2~24.04)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto=auto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-opencl --enable-libdrm --enable-libxml2 --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libharfbuzz --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libsvtav1 --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-vaapi --enable-amf --enable-libvpl --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc
  libavutil      59. 39.100 / 59. 39.100
  libavcodec    61. 19.101 / 61. 19.101
  libavformat    61.  7.100 / 61.  7.100
  libavdevice    61.  3.100 / 61.  3.100
  libavfilter    10.  4.100 / 10.  4.100
  libswscale      8.  3.100 /  8.  3.100
  libswresample  5.  3.100 /  5.  3.100
  libpostproc    58.  3.100 / 58.  3.100
[mpeg2video @ 0x62e8dae61c40] Invalid frame dimensions 0x0.
    Last message repeated 15 times
[mpegts @ 0x62e8dae41f00] PES packet size mismatch
[mpegts @ 0x62e8dae41f00] Packet corrupt (stream = 1, dts = 3151638590).
[mpegts @ 0x62e8dae41f00] PES packet size mismatch
[mpegts @ 0x62e8dae41f00] Packet corrupt (stream = 2, dts = 3151655870).
Input #0, mpegts, from 'file:/var/lib/jellyfin/data/livetv/recordings/Saturday Night Live/Season 50/Saturday Night Live S50E21 Scarlett Johansson; Bad Bunny.ts':
  Duration: 01:33:59.30, start: 29379.742556, bitrate: 6743 kb/s
  Program 3
  Stream #0:0[0x31]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv, top first), 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 59.94 tbr, 90k tbn
      Side data:
        cpb: bitrate max/min/avg: 19400000/0/0 buffer size: 7995392 vbv_delay: N/A
  Stream #0:1[0x34](eng): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 5.1(side), fltp, 384 kb/s
  Stream #0:2[0x35](spa): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, stereo, fltp, 128 kb/s
[AVHWDeviceContext @ 0x62e8dae75c00] Failed to initialise VAAPI connection: -1 (unknown libva error).
Device creation failed: -5.
[vist#0:0/mpeg2video @ 0x62e8dae91dc0] [dec:mpeg2video @ 0x62e8dae91f40] No device available for decoder: device type vaapi needed for codec mpeg2video.
[vist#0:0/mpeg2video @ 0x62e8dae91dc0] [dec:mpeg2video @ 0x62e8dae91f40] Hardware device setup failed for decoder: Input/output error
Error opening output file /var/cache/jellyfin/transcodes/8c8cb4b2f96a372e2fc1f9e5cc6954f9.m3u8.
Error opening output files: Input/output error

[nyanmisaka]

VAAPI cannot access iGPU video codecs in VMware virtual machines. A bare metal Linux installation is required.

[JdeFalconr]

VAAPI cannot access iGPU video codecs in VMware virtual machines. A bare metal Linux installation is required.

Thank you, I really appreciate the reply. Is that even with passthrough or SR-IOV? If I added a discrete GPU to the host and passed it through to the VM is there a reason that wouldn't work?

[nyanmisaka]

I suspect that this iGPU does not support SRIOV.

https://www.intel.com/content/www/us/en/...phics.html

[tollgate]

Just to add to this - I have an i5-4570T (Haswell). Jellyfin installed bare metal. VAAPI is all set up and working. But the quality when a hardware transcode occurs is pretty poor, especially on faster moving scenes. In fact I've turned off hardware transcoding and rely on software transcoding which just about keeps up.

[화정큐삼]

I recently tried Jellyfin hardware transcoding on a Haswell processor (i5-4210U, i3-4160T).
Jellyfin seems to have a default setting that uses the iHD driver instead of the i965 driver.
So, I forced the i965 driver to be used as follows:
export LIBVA_DRIVER_NAME=i965

This effectively forced Jellyfin's ffmpeg to use the i965 driver as the default.
However, Intel discontinued support for the i965 driver in 2019, which was when 4K video was first becoming widespread.
The i965 driver's limitation is that it cannot perform hardware transcoding of 4K video.
(It switches to software transcoding, resulting in nearly 100% CPU usage.)
Hardware transcoding from 1080p to 720p works without issue. While it would be nice if the iHD, which supports QSV, could function properly,
this also causes problems in Jellyfin.
Similarly, software transcoding is switched, resulting in nearly 100% CPU usage.
The reason for this issue is believed to be that both drivers lack the processing power for 4K video on Haswell's Gen7 iGPU.

The environment I tested was Synology DS918+ DSM 7.2.2 for the i5-4210U,
and Proxmox 8.4 (kernel 6.8.12-14-pve) for the i3-4160T.
For Proxmox, Jellyfin and libva must be compiled and installed on the host.
lxc also fails with i965.