SOLVED: Transcoding at some bitrates result in blank green video

[Jasper Michal]

This week I acquired an Arc A310 to use for transcoding on my jellyfin server. Previously I was using Intel integrated graphics. I got the new card installed, used DDU in safe mode, installed the new driver and everything appears to be fine.

However, when I have it transcode to an output of 40mbps or more, it results in the video being blank green (audio is still playing). Transcoding to anything less than 40mbps works fine, but 40 or above is green.

I fiddled around with it for a while and discovered that it only happens when transcoding and tone mapping. If I turn off tone mapping, it works normally. The issue is the same across all the clients I have tried, Chrome, Windows application and Android.

Possibly related, I don't think tone mapping is happening on the GPU, I think the CPU is doing it. My reasoning is 1, transcoding is obviously bottlenecked by the tone mapping and did not get any faster with the new GPU (transcoding without tone mapping did get faster) and 2, I get a massive spike in CPU usage when tone mapping, which doesn't happen when I'm transcoding without tone mapping. Is that possible? Is there some way of telling Jellyfin to use one or the other?

Specs:

i5-7700
32gb DDR5
Intel Arc A310
Windows 11 Pro 22H2
Native Windows Jellyfin Server
Jellyfin 10.8.13
ffmpeg version 5.1.4-Jellyfin

A screenshot of my transcode settings, and my ffmpeg log are attached.

Any wisdom would be appreciated!

[TheDreadPirate]

Try upgrading to jellyfin-ffmpeg6. You will have to manually replace ffmpeg5.

https://github.com/jellyfin/jellyfin-ffm...g/v6.0.1-1

CPU tone mapping is not happening on the CPU. That is currently not supported and, AFAIK, not possible to accidentally happen. Its possible the CPU usage is audio transcoding since that happens exclusively on the CPU. But without seeing your other transcode ffmpeg logs, I can't say why you don't see a lot of CPU usage on those streams. Its possible that only the video is being transcoded but not the audio, but I can't say for certain.

[Jasper Michal]

I successfully updated to 6.0.1, 40mbps and above still results in a green screen. It did introduce one new problem, now anything transcoded has weird green tinted elevated blacks on HDR displays (looks normal on SDR displays). It's also a little bit slower on average.

[TheDreadPirate]

Did you follow the instructions here?

https://jellyfin.org/docs/general/admini...ndows-host

There is a group policy change you need to make, though I am not entirely sure this is related to your problem.

---

Also, look into updating your BIOS.

[Jasper Michal]

I did use those instructions, group policy change and all. The BIOS occurred to me as well, but after updating to the latest versions it was the same. The board is an Asus H110M-C if that's of consequence.

[TheDreadPirate]

@nyanmisaka Are we missing anything?

[Jasper Michal]

For now I'm disabling the A310 and going back to the IGPU. Hopefully a solution will present itself at some point, being able to do more transcode streams simultaneously would be nice.

[Jasper Michal]

Some interesting data points from switching back and forth.

UHD 630 IGPU gets about 90 FPS transcoding + tone mapping 4K HDR blu ray rip to 1080p 20mbps.
Arc A310 gets about 50 FPS, and looks significantly worse.

Without tone mapping, the IPGU gets about 130 FPS and looks pretty much the same as the A310, which is pushing 280 fps.

It seems like something is going on with tone mapping on the A310, it's slower and looks worse than the IGPU, plus the green screen issue at higher bitrates.

[nyanmisaka]

Intel screwed up their Windows driver in 31.0.101.5186/31.0.101.5234. https://github.com/IGCIT/Intel-GPU-Commu...issues/680
Rolling back to the previous version 31.0.101.5085/31.0.101.5122 can fix this issue.

Also, the motherboards supported by the i7-7700 are too old to support Resizeable-BAR.
Arc GPUs require this feature to achieve maximum performance under certain circumstances.

[Jasper Michal]

Reverting to 31.0.101.5085 fixed it, thank you! Still not a huge increase in tone mapping performance (compared to transcoding without tone mapping), but none of the GPU engines or CPU cores show more than 30% usage, do you know what might be the bottleneck here?
It could be Re-BAR like you said, especially considering the motherboard only has PCIe 3 which is already bandwidth-limiting, but I'm just guessing.

Thanks for all the help!