Trickplay not being generated on 4K content

[pdieguez]

Hello all,

I have lots of content up to 1080p with Trickplay generated, however, I have noticed that my 4K content is not generating trickplay images. I'm running:

- Jellyfin 10.9.9 on a Docker Container with Armbian O.S.
- Orange Pi 5 SBC with Rockchip RK3588S chipset.
- I have Transcoding enabled with this chipset and it works properly when playing content

After doing some research on this forum, I came up with the following, after doing a fresh restart of my Docker container:

Logs after running the "Generate Trickplay Images" manually on schedule tasks:

Code:

[11:31:43] [INF] [27] Jellyfin.Server.Implementations.Trickplay.TrickplayManager: Creating trickplay files at 320 width, for /media/movies/The.Little.Mermaid.2023.2160p.WEB-DL.DDP5.1.Atmos.DV.HDR.H.265-FLUX.mkv [ID: 49ae49fb-6264-4985-cc59-23bfc0637a4f]
[11:31:43] [INF] [27] MediaBrowser.MediaEncoding.Encoder.MediaEncoder: Trickplay generation: /usr/lib/jellyfin-ffmpeg/ffmpeg -loglevel error -threads 1 -noautorotate -i file:"/media/movies/The.Little.Mermaid.2023.2160p.WEB-DL.DDP5.1.Atmos.DV.HDR.H.265-FLUX.mkv" -an -sn -vf "fps=0.10000000149011612,setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale=trunc(min(max(iw\,ih*(a*sar))\,320)/2)*2:trunc(ow/(a*sar)/2)*2,format=yuv420p" -threads 1 -c:v mjpeg -qscale:v 4 -f image2 "/cache/temp/a132cdaf531d4eed88ac17c333f1a248/%08d.jpg"
[11:32:23] [INF] [29] MediaBrowser.MediaEncoding.Encoder.MediaEncoder: Trickplay process unresponsive.
[11:32:23] [INF] [29] MediaBrowser.MediaEncoding.Encoder.MediaEncoder: Stopping trickplay extraction.
[11:32:24] [INF] [29] MediaBrowser.MediaEncoding.Encoder.MediaEncoder: Killing ffmpeg process
[11:32:24] [ERR] [29] MediaBrowser.MediaEncoding.Encoder.MediaEncoder: ffmpeg image extraction failed for /usr/lib/jellyfin-ffmpeg/ffmpeg -loglevel error -threads 1 -noautorotate -i file:"/media/movies/The.Little.Mermaid.2023.2160p.WEB-DL.DDP5.1.Atmos.DV.HDR.H.265-FLUX.mkv" -an -sn -vf "fps=0.10000000149011612,setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale=trunc(min(max(iw\,ih*(a*sar))\,320)/2)*2:trunc(ow/(a*sar)/2)*2,format=yuv420p" -threads 1 -c:v mjpeg -qscale:v 4 -f image2 "/cache/temp/a132cdaf531d4eed88ac17c333f1a248/%08d.jpg"
[11:32:24] [ERR] [29] Jellyfin.Server.Implementations.Trickplay.TrickplayManager: Error creating trickplay images.
MediaBrowser.Common.FfmpegException: ffmpeg image extraction failed for /usr/lib/jellyfin-ffmpeg/ffmpeg -loglevel error -threads 1 -noautorotate -i file:"/media/movies/The.Little.Mermaid.2023.2160p.WEB-DL.DDP5.1.Atmos.DV.HDR.H.265-FLUX.mkv" -an -sn -vf "fps=0.10000000149011612,setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale=trunc(min(max(iw\,ih*(a*sar))\,320)/2)*2:trunc(ow/(a*sar)/2)*2,format=yuv420p" -threads 1 -c:v mjpeg -qscale:v 4 -f image2 "/cache/temp/a132cdaf531d4eed88ac17c333f1a248/%08d.jpg"
  at MediaBrowser.MediaEncoding.Encoder.MediaEncoder.ExtractVideoImagesOnIntervalInternal(String inputArg, String filterParam, String vidEncoder, Nullable`1 outputThreads, Nullable`1 qualityScale, Nullable`1 priority, CancellationToken cancellationToken)
  at Jellyfin.Server.Implementations.Trickplay.TrickplayManager.RefreshTrickplayDataInternal(Video video, Boolean replace, Int32 width, TrickplayOptions options, CancellationToken cancellationToken)

Then, I tried running manually the following Trickplay extraction command on the container bash shell, and I could notice that the images are being generated on the associated folder, although very slowly, like 1 or 2 images per minute maximum, but no errors come up when running the command.

Code:

root@orangepi5:/# /usr/lib/jellyfin-ffmpeg/ffmpeg -loglevel error -threads 1 -noautorotate -i file:"/media/movies/The.Little.Mermaid.2023.2160p.WEB-DL.DDP5.1.Atmos.DV.HDR.H.265-FLUX.mkv" -an -sn -vf "fps=0.10000000149011612,setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale=trunc(min(max(iw\,ih*(a*sar))\,320)/2)*2:trunc(ow/(a*sar)/2)*2,format=yuv420p" -threads 1 -c:v mjpeg -qscale:v 4 -f image2 "/cache/temp/a132cdaf531d4eed88ac17c333f1a248/%08d.jpg"

Then I tried swiching "Enable hardware decoding" under Playback -> Trickplay settings, and it does not work at all, even when running the command manually.

Code:

[11:35:18] [INF] [25] Jellyfin.Server.Implementations.Trickplay.TrickplayManager: Creating trickplay files at 320 width, for /media/movies/The.Little.Mermaid.2023.2160p.WEB-DL.DDP5.1.Atmos.DV.HDR.H.265-FLUX.mkv [ID: 49ae49fb-6264-4985-cc59-23bfc0637a4f]
[11:35:18] [INF] [25] MediaBrowser.MediaEncoding.Encoder.MediaEncoder: Trickplay generation: /usr/lib/jellyfin-ffmpeg/ffmpeg -loglevel error -init_hw_device rkmpp=rk -hwaccel rkmpp -hwaccel_output_format drm_prime -noautorotate -i file:"/media/movies/The.Little.Mermaid.2023.2160p.WEB-DL.DDP5.1.Atmos.DV.HDR.H.265-FLUX.mkv" -noautoscale -an -sn -vf "fps=0.10000000149011612,setparams=color_primaries=bt2020:color_trc=smpte2084:colorspace=bt2020nc,scale_rkrga=w=320:h=132:format=p010,hwmap=derive_device=opencl:mode=read,tonemap_opencl=format=nv12:p=bt709:t=bt709:m=bt709:tonemap=bt2390:peak=100:desat=0:tradeoff=1,hwdownload,format=nv12" -threads 1 -c:v mjpeg -qscale:v 4 -f image2 "/cache/temp/183cd8c846d643db8b39b4915f560789/%08d.jpg"
[Parsed_scale_rkrga_2 @ 0xaaaadc0ef0e0] 'p010le' is not supported if RGA2 is requested
[Parsed_scale_rkrga_2 @ 0xaaaadc0ef0e0] Failed to configure output pad on Parsed_scale_rkrga_2
Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented
Error while processing the decoded data for stream #0:0
rga_api version 1.10.0_[8]
[11:35:19] [ERR] [15] Jellyfin.Server.Implementations.Trickplay.TrickplayManager: Error creating trickplay images.
System.ArgumentException: Can't create trickplay from 0 images.
  at Jellyfin.Server.Implementations.Trickplay.TrickplayManager.CreateTiles(List`1 images, Int32 width, TrickplayOptions options, String outputDir)
  at Jellyfin.Server.Implementations.Trickplay.TrickplayManager.RefreshTrickplayDataInternal(Video video, Boolean replace, Int32 width, TrickplayOptions options, CancellationToken cancellationToken)

When running on bash with hardware decoding:

Code:

root@orangepi5:/# /usr/lib/jellyfin-ffmpeg/ffmpeg -loglevel error -init_hw_device rkmpp=rk -hwaccel rkmpp -hwaccel_output_format drm_prime -noautorotate -i file:"/media/movies/The.Little.Mermaid.2023.2160p.WEB-DL.DDP5.1.Atmos.DV.HDR.H.265-FLUX.mkv" -noautoscale -an -sn -vf "fps=0.10000000149011612,setparams=color_primaries=bt2020:color_trc=smpte2084:colorspace=bt2020nc,scale_rkrga=w=320:h=132:format=p010,hwmap=derive_device=opencl:mode=read,tonemap_opencl=format=nv12:p=bt709:t=bt709:m=bt709:tonemap=bt2390:peak=100:desat=0:tradeoff=1,hwdownload,format=nv12" -threads 1 -c:v mjpeg -qscale:v 4 -f image2 "/cache/temp/183cd8c846d643db8b39b4915f560789/%08d.jpg"
rga_api version 1.10.0_[8]
[Parsed_scale_rkrga_2 @ 0xaaaae71790e0] 'p010le' is not supported if RGA2 is requested
[Parsed_scale_rkrga_2 @ 0xaaaae71790e0] Failed to configure output pad on Parsed_scale_rkrga_2
Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented
Error while processing the decoded data for stream #0:0

Docker syntax used for running Jellyfin:

Code:

docker run -e TZ=America/Sao_Paulo -d \
--privileged \
--name=jellyfin \
--volume /srv/dev-disk-by-uuid-5e80b78e-0822-4b39-bec6-e2eb1ca9e01d/jellyfin/config:/config \
--volume /srv/dev-disk-by-uuid-5e80b78e-0822-4b39-bec6-e2eb1ca9e01d/jellyfin/cache:/cache \
--volume /srv/dev-disk-by-uuid-5e80b78e-0822-4b39-bec6-e2eb1ca9e01d/jellyfin/media:/media \
--volume /srv/dev-disk-by-uuid-4E1AEA7B1AEA6007/src/torrents:/media-torrents \
--user 0:0 \
--net=host \
--restart=unless-stopped \
`for dev in dri dma_heap mali0 rga mpp_service \
  iep mpp-service vpu_service vpu-service \
  hevc_service hevc-service rkvdec rkvenc vepu h265e ; do \
  [ -e "/dev/$dev" ] && echo " --device /dev/$dev"; \
done` \
jellyfin/jellyfin

And that's it. Any help appreciated. 
Thank you all very much.

[TheDreadPirate]

It may be failing because the tone mapping is not working.

Check out this github issue.

https://github.com/jellyfin/jellyfin/issues/12010

This issue is for tone mapping video playback but the same parameters are used to tone map HDR video to SDR jpegs.

@nyanmisaka 's comment here may be what you need.

https://github.com/jellyfin/jellyfin/iss...2154386856

[pdieguez]

Hi there,

How can I rest assured that Tone Mapping is not working?
I can normally play 4K content transcoded to 1080p, 720p and 480p and I have also downloaded the short video provided on the Github issue you provided me, and I could normally play that transcoded to lower resolutions as well.

Besided that, I am using the older version of 

Code:

libmali-valhall-g610-g13p0-gbm_1.9-1_arm64.deb

since the beginning of my installation.

I believe that my transcoding is working just fine.

[gnattu]

I think it is just your board is too slow for this task and therefore it timed out. We may add hardware accelerated trickplay pipeline for RK3588 in future releases but the time frame is still unknown.

[pdieguez]

Well, taking that into account, is there anyway to let it run even though very slowly?

[gnattu]

Go to your config folder, open system.xml and look for ImageExtractionTimeoutMs. It is 0 by default which means 10 seconds for SDR video and 20 seconds for HDR videos, which is theoretically long enough for most CPU for most videos. You can set it to a larger value like 30000 so that it will wait for 30 seconds.