SOLVED: Jellyfin becomes unresponsive when transcoding

[Henkow]

ffmpeg log could be informative

recommended tests
speed test the transcode target drive https://www.cyberciti.biz/faq/howto-linu...dd-command
check resources usage with iotop during playback attempt https://www.geeksforgeeks.org/iotop-comm...h-examples

if ram is available and cpu usage is low; suspect ssd
move forward from the ffmpeg log and the recommended tests

I also suspected the SSD at first, I did run an extended S.M.A.R.T. test on it and it looked good. I also tried to write and read to it concurrently and achieved speeds in excess of 100MB/s both ways.

Result from the speed test:
> dd if=/dev/zero of=/tmp/test1.img bs=1G count=1 oflag=dsync
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.679026 s, 1.6 GB/s
> dd if=/dev/zero of=/tmp/test2.img bs=512 count=1000 oflag=dsync
1000+0 records in
1000+0 records out
512000 bytes (512 kB, 500 KiB) copied, 0.00308955 s, 166 MB/s

Output from iotop: https://ibb.co/TcxVbwf. It took about a minute or so before anything registrered on READ (despite this the playback was fine?). Then it jumped between reading and not reading anything between each iotop refresh. I suppose that makes sense as it receives the files in chunks? The playback seems to consistently stall around 2 minutes in on firefox and 40 seconds in on chrome.

I can't say for certain, but your server might just be overwhelmed.  Two transcodes without a GPU on an old CPU with only 4 cores.

Also, what are you playing these videos on that doesn't support H264?

I should have been more clear regarding the transcoding, only the audio is trancoded (direct streaming). I'm playing the movie back in firefox, I've tried chrome as well but firefox have worked better for me in the past. Around a year ago when I set up Jellyfin I was able to transcode 4 movies (blu-ray remuxes) concurrently without any issues. I tend to use Kodi for playback most of the time. I only noticed this issue now as I gave remote access to a family member and I noticed I couldn't access the Web UI while they were streaming. I was then able to reproduce it locally (I've turned off the reverse proxy for the time being).

I updated the server log as I ran a more controlled test with just a single movie.

[16:26] I start playback
[16:28] I open a new tab, which never load. At this time the movie also freeze. After about 90 seconds later it resumes playback but it then starts to play intermittently. (Edit: it appears the movie will stall regardless if I open a new tab or not.)
[16:32] I exit playback by pressing the back symbol in the Web UI.
[16:34] I manually kill ffmpeg which have now transcoded 37 minutes of the movie.
[16:37] Jellyfin becomes responsive again.