2024-10-03, 06:05 AM
(This post was last modified: 2024-10-11, 10:32 PM by Duvel. Edited 8 times in total.)
Hi
Running 10.9.11 in docker : jellyfin/jellyfin:latest on a Ubuntu 22.04 VM.
I've got a mature server with tenths of users running for years with no issues.
This morning that server was KO because the system disk was full
I've got a 100TB SDD for the system and its where Jellyfin files lies, including the transcode directory
There were more than 13k files in the directory and all files were created on yesterday and 2 days ago.
like:
-rw-r--r-- 1 jellyfin jellyfin 2443293 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b35.mp4
-rw-r--r-- 1 jellyfin jellyfin 2317467 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b360.mp4
-rw-r--r-- 1 jellyfin jellyfin 2652326 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b361.mp4
-rw-r--r-- 1 jellyfin jellyfin 3282917 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b362.mp4
-rw-r--r-- 1 jellyfin jellyfin 2614603 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b363.mp4
-rw-r--r-- 1 jellyfin jellyfin 2999447 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b364.mp4
-rw-r--r-- 1 jellyfin jellyfin 2785210 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b365.mp4
-rw-r--r-- 1 jellyfin jellyfin 2418162 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b366.mp4
-rw-r--r-- 1 jellyfin jellyfin 2461479 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b367.mp4
-rw-r--r-- 1 jellyfin jellyfin 2489552 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b368.mp4
-rw-r--r-- 1 jellyfin jellyfin 2464962 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b369.mp4
-rw-r--r-- 1 jellyfin jellyfin 2285016 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b36.mp4
-rw-r--r-- 1 jellyfin jellyfin 2456811 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b370.mp4
-rw-r--r-- 1 jellyfin jellyfin 2520562 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b371.mp4
-rw-r--r-- 1 jellyfin jellyfin 2570875 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b372.mp4
-rw-r--r-- 1 jellyfin jellyfin 2911734 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b373.mp4
-rw-r--r-- 1 jellyfin jellyfin 2606474 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b374.mp4
-rw-r--r-- 1 jellyfin jellyfin 2675693 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b375.mp4
-rw-r--r-- 1 jellyfin jellyfin 2890279 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b376.mp4
-rw-r--r-- 1 jellyfin jellyfin 2769607 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b377.mp4
-rw-r--r-- 1 jellyfin jellyfin 2905585 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b378.mp4
-rw-r--r-- 1 jellyfin jellyfin 2449224 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b379.mp4
-rw-r--r-- 1 jellyfin jellyfin 1690650 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b37.mp4
-rw-r--r-- 1 jellyfin jellyfin 2111243 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b380.mp4
-rw-r--r-- 1 jellyfin jellyfin 1927557 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b381.mp4
-rw-r--r-- 1 jellyfin jellyfin 2007292 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b382.mp4
-rw-r--r-- 1 jellyfin jellyfin 2684367 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b383.mp4
-rw-r--r-- 1 jellyfin jellyfin 2396827 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b384.mp4
-rw-r--r-- 1 jellyfin jellyfin 2360938 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b385.mp4
-rw-r--r-- 1 jellyfin jellyfin 2843802 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b386.mp4
-rw-r--r-- 1 jellyfin jellyfin 3176584 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b387.mp4
-rw-r--r-- 1 jellyfin jellyfin 2668085 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b388.mp4
-rw-r--r-- 1 jellyfin jellyfin 2994367 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b389.mp4
-rw-r--r-- 1 jellyfin jellyfin 2063018 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b38.mp4
-rw-r--r-- 1 jellyfin jellyfin 2744151 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b390.mp4
-rw-r--r-- 1 jellyfin jellyfin 2591348 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b391.mp4
-rw-r--r-- 1 jellyfin jellyfin 2768626 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b392.mp4
-rw-r--r-- 1 jellyfin jellyfin 2782224 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b393.mp4
-rw-r--r-- 1 jellyfin jellyfin 2853041 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b394.mp4
-rw-r--r-- 1 jellyfin jellyfin 3026344 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b395.mp4
-rw-r--r-- 1 jellyfin jellyfin 3116120 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b396.mp4
-rw-r--r-- 1 jellyfin jellyfin 2333962 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b397.mp4
-rw-r--r-- 1 jellyfin jellyfin 2372371 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b398.mp4
I purged the whole directory and regained 30TB
2 days ago (1st october evening) I found that the jellyfin server was slow to respond, while I had 5 users watching something on it. I had a look at the log and did not find anything obvious. I thus decided to restart the container. I rarely have to do this. The server answered much better after that.
All the transcode files were newer than that restart, meaning that it created 13k transcode files in 24 hours and never deleted them itself.
There's obviously a big issue somewhere and I'd like that you help me to find the root cause in my log files (I'd like to avoid to post them all publicly if possible).
Cheers
Running 10.9.11 in docker : jellyfin/jellyfin:latest on a Ubuntu 22.04 VM.
I've got a mature server with tenths of users running for years with no issues.
This morning that server was KO because the system disk was full
Quote:tom@www-prd:~$ df
Filesystem 1K-blocks Used Available Use% Mounted on
tmpfs 1224444 5560 1218884 1% /run
/dev/sda2 102656700 98453460 0 100% /
tmpfs 6122208 28 6122180 1% /dev/shm
tmpfs 5120 0 5120 0% /run/lock
/dev/sdb1 1073216516 545110924 528105592 51% /var/www
/dev/sdc1 16104039404 15023022980 1081016424 94% /media/plex
tmpfs 1224440 60 1224380 1% /run/user/128
I've got a 100TB SDD for the system and its where Jellyfin files lies, including the transcode directory
There were more than 13k files in the directory and all files were created on yesterday and 2 days ago.
like:
-rw-r--r-- 1 jellyfin jellyfin 2443293 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b35.mp4
-rw-r--r-- 1 jellyfin jellyfin 2317467 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b360.mp4
-rw-r--r-- 1 jellyfin jellyfin 2652326 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b361.mp4
-rw-r--r-- 1 jellyfin jellyfin 3282917 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b362.mp4
-rw-r--r-- 1 jellyfin jellyfin 2614603 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b363.mp4
-rw-r--r-- 1 jellyfin jellyfin 2999447 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b364.mp4
-rw-r--r-- 1 jellyfin jellyfin 2785210 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b365.mp4
-rw-r--r-- 1 jellyfin jellyfin 2418162 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b366.mp4
-rw-r--r-- 1 jellyfin jellyfin 2461479 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b367.mp4
-rw-r--r-- 1 jellyfin jellyfin 2489552 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b368.mp4
-rw-r--r-- 1 jellyfin jellyfin 2464962 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b369.mp4
-rw-r--r-- 1 jellyfin jellyfin 2285016 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b36.mp4
-rw-r--r-- 1 jellyfin jellyfin 2456811 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b370.mp4
-rw-r--r-- 1 jellyfin jellyfin 2520562 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b371.mp4
-rw-r--r-- 1 jellyfin jellyfin 2570875 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b372.mp4
-rw-r--r-- 1 jellyfin jellyfin 2911734 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b373.mp4
-rw-r--r-- 1 jellyfin jellyfin 2606474 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b374.mp4
-rw-r--r-- 1 jellyfin jellyfin 2675693 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b375.mp4
-rw-r--r-- 1 jellyfin jellyfin 2890279 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b376.mp4
-rw-r--r-- 1 jellyfin jellyfin 2769607 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b377.mp4
-rw-r--r-- 1 jellyfin jellyfin 2905585 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b378.mp4
-rw-r--r-- 1 jellyfin jellyfin 2449224 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b379.mp4
-rw-r--r-- 1 jellyfin jellyfin 1690650 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b37.mp4
-rw-r--r-- 1 jellyfin jellyfin 2111243 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b380.mp4
-rw-r--r-- 1 jellyfin jellyfin 1927557 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b381.mp4
-rw-r--r-- 1 jellyfin jellyfin 2007292 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b382.mp4
-rw-r--r-- 1 jellyfin jellyfin 2684367 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b383.mp4
-rw-r--r-- 1 jellyfin jellyfin 2396827 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b384.mp4
-rw-r--r-- 1 jellyfin jellyfin 2360938 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b385.mp4
-rw-r--r-- 1 jellyfin jellyfin 2843802 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b386.mp4
-rw-r--r-- 1 jellyfin jellyfin 3176584 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b387.mp4
-rw-r--r-- 1 jellyfin jellyfin 2668085 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b388.mp4
-rw-r--r-- 1 jellyfin jellyfin 2994367 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b389.mp4
-rw-r--r-- 1 jellyfin jellyfin 2063018 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b38.mp4
-rw-r--r-- 1 jellyfin jellyfin 2744151 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b390.mp4
-rw-r--r-- 1 jellyfin jellyfin 2591348 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b391.mp4
-rw-r--r-- 1 jellyfin jellyfin 2768626 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b392.mp4
-rw-r--r-- 1 jellyfin jellyfin 2782224 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b393.mp4
-rw-r--r-- 1 jellyfin jellyfin 2853041 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b394.mp4
-rw-r--r-- 1 jellyfin jellyfin 3026344 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b395.mp4
-rw-r--r-- 1 jellyfin jellyfin 3116120 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b396.mp4
-rw-r--r-- 1 jellyfin jellyfin 2333962 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b397.mp4
-rw-r--r-- 1 jellyfin jellyfin 2372371 Oct 2 21:29 f27db32e08a850103ccdaf0b5e93e9b398.mp4
I purged the whole directory and regained 30TB
2 days ago (1st october evening) I found that the jellyfin server was slow to respond, while I had 5 users watching something on it. I had a look at the log and did not find anything obvious. I thus decided to restart the container. I rarely have to do this. The server answered much better after that.
All the transcode files were newer than that restart, meaning that it created 13k transcode files in 24 hours and never deleted them itself.
There's obviously a big issue somewhere and I'd like that you help me to find the root cause in my log files (I'd like to avoid to post them all publicly if possible).
Cheers