Jellyfin Forum
AV1 Transcoding Playback Failure - Printable Version

+- Jellyfin Forum (https://forum.jellyfin.org)
+-- Forum: Support (https://forum.jellyfin.org/f-support)
+--- Forum: Troubleshooting (https://forum.jellyfin.org/f-troubleshooting)
+--- Thread: AV1 Transcoding Playback Failure (/t-av1-transcoding-playback-failure)



AV1 Transcoding Playback Failure - conor - 2025-11-26

When transcoding to AV1 the video is in a non-standard container and the Jellyfin ffmpeg log gives the below warning.

[mpegts @ 0x7f78e0220580] Stream 0, codec av1, is muxed as a private data stream and may not be recognized upon reading.

Audio plays but there's no video stream recognised in Kodi or VLC with the Jellyfin generated ffmpeg command.

Changing hls_segment_type to fmp4 gives a stream that plays successfully in VLC.

AV1 transcoding worked in 10.10 for me, has something changed in 10.11?

Original non-working ffmpeg command (AMD 780m GPU).

Code:
/usr/lib/jellyfin-ffmpeg/ffmpeg \
  -analyzeduration 200M \
  -probesize 1G \
  -f matroska \
  -init_hw_device drm=dr:/dev/dri/renderD128 \
  -init_hw_device vaapi=va@dr \
  -init_hw_device vulkan=vk@dr \
  -filter_hw_device vk \
  -hwaccel vaapi \
  -hwaccel_output_format vaapi \
  -noautorotate \
  -i "file:/media/test/input.mkv" \
  -noautoscale \
  -map_metadata -1 \
  -map_chapters -1 \
  -threads 0 \
  -map 0:0 -map 0:1 -map -0:s \
  -codec:v:0 av1_vaapi \
  -rc_mode VBR \
  -b:v 5666112 \
  -maxrate 5666112 \
  -bufsize 11332224 \
  -force_key_frames:0 "expr:gte(t,n_forced*3)" \
  -vf "setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale_vaapi=format=nv12" \
  -codec:a:0 ac3 \
  -ac 6 \
  -ab 256000 \
  -ar 48000 \
  -copyts \
  -avoid_negative_ts disabled \
  -max_muxing_queue_size 2048 \
  -f hls \
  -max_delay 5000000 \
  -hls_time 3 \
  -hls_segment_type mpegts \
  -hls_playlist_type vod \
  -hls_list_size 0 \
  -start_number 0 \
  -hls_segment_filename "/media/encode/test2/5469b5100c67ae2c9b18e3ee80d56c8b%d.ts" \
  -y "/media/encode/test2/5469b5100c67ae2c9b18e3ee80d56c8b.m3u8"