+- Jellyfin Forum (https://forum.jellyfin.org)
+-- Forum: Support (https://forum.jellyfin.org/f-support)
+--- Forum: Troubleshooting (https://forum.jellyfin.org/f-troubleshooting)
+--- Thread: Jellyfin deleted 2.5tb of my personal data without prompt or asking. (/t-jellyfin-deleted-2-5tb-of-my-personal-data-without-prompt-or-asking)
Pages:
1
2
Jellyfin deleted 2.5tb of my personal data without prompt or asking. - KidDeath76 - 2023-08-30
I can not possibly express how angry and upset I am right now, but I will absolutely keep it civil and fact-focused as much as I can.
A few days ago, I couldn't get thumbnails to load from one of my folders (user error - I simply hadn't checked the box to download them). But in a moment of stupid troubleshooting, I tried relocating the cache folder to the root of my library folder (in its own named folder, simply next to other folders I was scanning!) as a troubleshooting method. I had read a few times there could be various issues with the default location due to permissions issues accessing the C: drive. I thought relocating the cache to a different drive would enable Jellyfin to access the library and have permission to access the thumbnails.
Without warning, the next morning, Jellyfin deleted MY ENTIRE 3TB LIBRARY OF MEDIA. Over 20 years of storage.
As the logs can show you, here is what it says:
[2023-08-27 09:53:47.876 -07:00] [INF] [34] Emby.Server.Implementations.ScheduledTasks.TaskManager: "Optimize database" Completed after 0 minute(s) and 0 seconds
[2023-08-27 09:53:47.885 -07:00] [INF] [34] Emby.Server.Implementations.ScheduledTasks.TaskManager: ExecuteQueuedTasks
[2023-08-27 09:53:50.485 -07:00] [INF] [34] Emby.Server.Implementations.ScheduledTasks.TaskManager: "IntervalTrigger" fired for task: "Clean Cache Directory"
[2023-08-27 09:53:50.490 -07:00] [INF] [34] Emby.Server.Implementations.ScheduledTasks.TaskManager: Queuing task "DeleteCacheFileTask"
[2023-08-27 09:53:50.495 -07:00] [INF] [38] Emby.Server.Implementations.ScheduledTasks.TaskManager: Executing "Clean Cache Directory"
[2023-08-27 09:53:59.649 -07:00] [INF] [50] Emby.Server.Implementations.ScheduledTasks.TaskManager: "IntervalTrigger" fired for task: "Refresh Guide"
[2023-08-27 09:53:59.654 -07:00] [INF] [50] Emby.Server.Implementations.ScheduledTasks.TaskManager: Queuing task "RefreshGuideScheduledTask"
[2023-08-27 09:53:59.659 -07:00] [INF] [34] Emby.Server.Implementations.ScheduledTasks.TaskManager: Executing "Refresh Guide"
[2023-08-27 09:54:02.757 -07:00] [INF] [50] Emby.Server.Implementations.LiveTv.LiveTvManager: Refreshing guide with 7 days of guide data
[2023-08-27 09:54:02.761 -07:00] [INF] [50] Emby.Server.Implementations.ScheduledTasks.TaskManager: "Refresh Guide" Completed after 0 minute(s) and 3 seconds
[2023-08-27 09:54:02.766 -07:00] [INF] [50] Emby.Server.Implementations.ScheduledTasks.TaskManager: ExecuteQueuedTasks
[2023-08-27 09:54:26.885 -07:00] [INF] [50] Emby.Server.Implementations.ScheduledTasks.TaskManager: "IntervalTrigger" fired for task: "Scan Media Library"
[2023-08-27 09:54:26.890 -07:00] [INF] [50] Emby.Server.Implementations.ScheduledTasks.TaskManager: Queuing task "RefreshMediaLibraryTask"
[2023-08-27 09:54:26.896 -07:00] [INF] [34] Emby.Server.Implementations.ScheduledTasks.TaskManager: Executing "Scan Media Library"
[2023-08-27 09:54:26.901 -07:00] [INF] [34] Emby.Server.Implementations.IO.LibraryMonitor: Stopping directory watching for path "[Removed for privacy]
[2023-08-27 09:54:26.940 -07:00] [INF] [34] Emby.Server.Implementations.Library.LibraryManager: Validating media library
[2023-08-27 09:55:03.827 -07:00] [INF] [47] Emby.Server.Implementations.Library.LibraryManager: Removing item, Type: "Movie", Name: " [Removed for privacy]
[2023-08-27 09:55:03.833 -07:00] [INF] [24] Emby.Server.Implementations.Library.LibraryManager: Removing item, Type: "Movie", Name: " [Removed for privacy]
[2023-08-27 09:55:02.397 -07:00] [INF] [53] Emby.Server.Implementations.Library.LibraryManager: Removing item, Type: "PhotoAlbum", Name:
[2023-08-27 09:55:04.804 -07:00] [INF] [28] Emby.Server.Implementations.Library.LibraryManager: Removing item, Type: "Folder", Name:
etc... it goes on for 3tb.
Unfortunately, lots of personal and company data lies in these logs, so I'm not willing to post them fully (although I could try to remove it to post them, but it would take me forever and I don't think would matter. The damage is done.)
I used data recovery software immediately that night when I noticed, both free and paid, and was able to recover only about 1/2 or maybe 2/3rds at best of my personal data.
Now, I work with computers for a living and I am extremely tech savvy. If this can happen to me, it has to be able to happen to any random uneducated consumer.
My question is this: why the heck did this happen? How in the world is it possible Jellyfin is able to do this? WHY would this be programmed this way? I am absolutely at a loss for words here. Literal family photos and priceless documents from my dead mother were deleted. I can not possibly express my dissatisfaction and anger, although I commit to keeping it civil on these forums for sake of everyone's learning and knowledge.
Thank you,
KD
RE: Jellyfin deleted 2.5tb of my personal data without prompt or asking. - skribe - 2023-08-30
Nothing in that log snippet indicates that your media was deleted by Jellyfin. Jellyfin scanned your library and removed items from the Jellyfin database--likely because they were not found on your storage.
You will need further troubleshooting steps to determine why the media was actually deleted. I would check any system logs that might be relevant.
RE: Jellyfin deleted 2.5tb of my personal data without prompt or asking. - sevenrats - 2023-08-30
Jellyfin will only delete files within the directory that is specified as its cache. If you actually set your cache to the path of your library then the cache cleanup task probably did indeed delete your data (If so, why did you do this?). If you simply put the cache directory within the library, its not possible for JF to have deleted files outside of the cache path. My first guess would be that your mount simply failed, Jellyfin noticed the directory was empty and cleaned up the database for you, which is what these logs show, and that your data is intact on the drive that unmounted.
Remember that having a single copy of data is the same as having no copies. Its simply a matter of time until drive failure wipes it out. You must create a backup of your critical data.
RE: Jellyfin deleted 2.5tb of my personal data without prompt or asking. - bitmap - 2023-08-30
As a not-affiliated member of the community, I want to say two things: first, I'm sorry this happened to you. I don't know as though that thought has been expressed yet. I hold no culpability and I don't think saying "I'm sorry" holds any moral or ethical weight in role in the loss of your files. I couldn't have had anything to do with it according to the laws of space and time.
Second, the responses from the mods are very diplomatic and measured. That's appreciated an expected, they're representing the community. But your anger is misdirected. You started off with the right sentence: you made a bad decision in moving the cache folder, which resulted in this catastrophe. I understand the anger and despair, but it's a learning experience.
Quote:If this can happen to me, it has to be able to happen to any random uneducated consumer.
No, the uneducated consumer isn't likely to a) set up Jellyfin in the first place or b) move the cache directory. This is the first time I've seen a "Jellyfin deleted all of my media" topic and I've been running the software and participating in the community for several years now (~4-5 if I remember right). Have I deleted my own shit? Absolutely. Was I pissed? You betcha. Was it 100% my fault? Damn straight.
Quote:How in the world is it possible Jellyfin is able to do this? WHY would this be programmed this way?
As @sevenrats stated, if you set your cache directory as your media library, well, cache is by definition temporary and designed to be deleted automatically at whatever intervals the software deems necessary for proper functioning. The program wasn't designed to have media and cache hosted in the same directories, however. It's not programmed this way, you deliberately altered something in a way that changed the way the program functioned -- in a way that you admit you knew was probably dangerous -- and the worst case scenario happened.
I just want to reiterate: I'm sorry this happened to you. IT SUCKS. There's no replacing data that can't be recovered. Please try to pivot from anger to action and use it as a learning experience. The software is not to blame, nor are the folks in this community. The next time you struggle with an open source tool, please engage with the community. The fine folks like @skribe and sevenrats (already tagged you) as well as @TheDreadPirate (who removed his post, but is always professional, knowledgeable, and helpful) are here to try and help -- except me, I'm kind of an asshole.
Jellyfin, in particular, has a community that has been less toxic than most of the others I've tried to engage with and it's why I've come to know as much (which is so, so little) about the software. Folks are here to help, but you have to reach out before you get to this point. Nobody wants to see you angry, disappointed, and likely to spurn the tool and community for a bad decision that none of us -- nor Jellyfin -- had anything to do with.
RE: Jellyfin deleted 2.5tb of my personal data without prompt or asking. - Oddstr13 - 2023-09-01
STOP WRITING DATA TO THE DRIVE
Alright, so, the first part of recovering data in a scenario like this, where data has been accidentally deleted, is to stop writing data to the drive.
Unmount it in some way. if it is used by windows in any way, stop using the computer.
Then you need to evaluate how important this data is to you.
If it is extremely important, I would suggest you consider hiring a data recovery specialist, this will cost $$$$$, but may be worth it depending on how much you value the lost data.
If you should decide to attempt to tackle the problem yourself, then you should go buy an additional drive that can hold at least all the lost data, I'd suggest a larger disk than the one you currently have.
Preferably you would buy two drives. So that you can make an additional copy of your data.
I have in the past used TestDisk and PhotoRec to recover some parts of lost data.
Such data tends to end up as a jumbled mess of numbered files, as the file names are stored separately from the files on the disk, and it is this metadata that has been accidentally deleted.
Disk fragmentation would likely pose a problem for such recovery, as then the files aren't stored sequentially on the disk, but whatever you do, do not run a disk defragment now. That would almost certainly destroy your chances of recovering anything.
I won't be able to provide you with any more detailed instructions or guidance than this.
If you are near Austin Texas (or in the US) and want to go the specialist route, I suggest you consider Louis Rossmann's services https://rossmanngroup.com/data-recovery-service-austin/
I have no affiliation with Rossmann other than being a subscriber of his on the YouTube channel.
Data-loss sucks, no matter the cause, so my condolences on that.
PS:
Legal disclaimer that I am not responsible for anything anyone decides to do acting on any information provided etc. etc.
RE: Jellyfin deleted 2.5tb of my personal data without prompt or asking. - ric - 2024-03-27
May I be bold and say that I am really surprised as a new user to jellyfin that with hardware transcoding there is even a need for hard drive cache. Ive been using omx compiled version of FFMPEG for years with tvheadend and it works amazing in transcoding a TV channel TS Mpeg2 stream's in real time over VPN with no hard drive caching and low CPU on a PI2!!!!, and with no fancy options.
The other thing i would say is that if there is every a need for a cache directory then I think in most cases it would make sense to put it on the largest drive (of course the perfect solution would be a ram disk if you have the space). if jellyfin removes its cache files then why is it set to remove everything. surely from a neat coding POV it should just remove all cache 'pattern' files not just *.*
Much better would be for JF to use a virtual interface for one continuous transcoded TS stream. Also in most cases audio would never need to be transcoded.
TVH Transcoding HW on PI
------------------------
/usr/bin/ffmpeg -i - -f mpegts -c:a copy -c:v h264_omx -b:v 768k pipe:1
ffmpeg version 3.2.10-1~deb9u1~bpo8+1 Copyright © 2000-2018 the FFmpeg developers
built with gcc 4.9.2 (Debian 4.9.2-10)
configuration: --prefix=/usr --extra-version='1~deb9u1~bpo8+1' --toolchain=hardened --libdir=/usr/lib/arm-linux-gnueabihf --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --disable-libebur128 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
libavutil 55. 34.101 / 55. 34.101
libavcodec 57. 64.101 / 57. 64.101
libavformat 57. 56.101 / 57. 56.101
libavdevice 57. 1.100 / 57. 1.100
libavfilter 6. 65.100 / 6. 65.100
libavresample 3. 1. 0 / 3. 1. 0
libswscale 4. 2.100 / 4. 2.100
libswresample 2. 3.100 / 2. 3.100
libpostproc 54. 1.100 / 54. 1.100
DEV.LS h264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_vdpau ) (encoders: libx264 libx264rgb h264_omx h264_vaapi )
yeap version 3 and still going strong......
(./configure --enable-gpl --enable-nonfree --enable-mmal --enable-omx --enable-omx-rpi)
RE: Jellyfin deleted 2.5tb of my personal data without prompt or asking. - Efficient_Good_5784 - 2024-03-27
(2024-03-27, 09:17 AM)ric Wrote: The other thing i would say is that if there is every a need for a cache directory then I think in most cases it would make sense to put it on the largest drive (of course the perfect solution would be a ram disk if you have the space).Can you explain your reasoning for this statement?
I may be missing or don't know something, but why does it have to be on the largest drive? Were you referring to the fastest drive?
Obviously you need a drive with enough space to hold the amount of cache that will be generated within a given time frame, but I've been thinking about this and don't see why it needs to be on the largest drive on a system.
(2024-03-27, 09:17 AM)ric Wrote: if jellyfin removes its cache files then why is it set to remove everything.One of the points presented in this thread on why OP lost his media is because they possibly told Jellyfin that their media folder(s) were cache folders.
Cache is considered safe to destroy as long as the original files the cache was derived from still exist. You can always recreate it.
Jellyfin expects to start on an empty cache folder and build it for itself. As in, it expects its cache folder to be exclusively for itself to use and no one else.
So since Jellyfin has that expectation, it modifies/deletes all the files in the cache folder path as it pleases because it has no reason to doubt that others are using it.
The cache folder grows slowly holding images and metadata that each client opens in the server. The idea being that if an item was recently used, maybe other clients will use it in the near future, so it will have it cached to serve it faster in the future.
In the dashboard, there's a task set by default to clean the cache directory.
It's important to regularly clean out the cache directory because you don't want old cached data that doesn't get used anymore to take up space. You also might have cached something that no longer exists in the server, so it makes sense to delete that too.
One of the things it looks out for to clean is folders and files that doesn't make sense to how it normally creates cached items. Again, it does this because it expects that the directory was created by itself and that the important data lives elsewhere.
Same thing with the transcode directory. It regularly cleans this directory out since transcodes can be remade again.
Basically, just don't link either the cache or transcode folder to important directories on your system and you'll avoid issues.
(2024-03-27, 09:17 AM)ric Wrote: surely from a neat coding POV it should just remove all cache 'pattern' files not just *.*Have you seen what the cache folder looks like?
If you run a Jellyfin server, go and look at it. Open it up in a file browser or a system shell.
You'll see that all the cache items that Jellyfin creates have seemingly random names. All the cache files don't share a pattern to distinguish them as cache files.
The only distinguishing characteristic is that they all reside in the cache folder.
RE: Jellyfin deleted 2.5tb of my personal data without prompt or asking. - TheDreadPirate - 2024-03-27
(2024-03-27, 11:12 AM)Efficient_Good_5784 Wrote: One of the points presented in this thread on why OP lost his media is because they possibly told Jellyfin that their media folder(s) were cache folders.
Cache is considered safe to destroy as long as the original files the cache was derived from still exist. You can always recreate it.
IIRC, the OP put their cache directory at or above the directory their media was in to try to troubleshoot some permissions issue. So when the scheduled job to recursively clear the cache ran it also deleted their media.
RE: Jellyfin deleted 2.5tb of my personal data without prompt or asking. - ric - 2024-03-28
I can only talk from a PI perspective, but caching not just the transcoding files but also the var and usr cache files on the SDcard is going kill performance and the Card eventually
. Ive already linked the directories to a separate drive. Ive also redirected the transcoding FFMPEG to a shell script which strips out the key options and I pipes them through to ffmpeg the way which works best for me and keeps it simple... its a shame its caches the whole file, better would be just buffer a few seconds in real time and let the client do any video caching. Kodi does this well for TVHeadend.Im happy now!
server log file injecting my options into the stream:
0 -sn -codec:v:0 libx264 -pix_fmt yuv420p -preset veryfast -crf 23 -maxrate 800000 -bufsize 1600000 -x264opts:0 subme=0:me_range=4:rc_lookahead=10:me=dia:no_chroma_me:8x8dct=0:partitions=none -force_key_frames "expr:gte(t,n_forced*3)" -vf "scale=trunc(min(max(iw\,ih*dar)\,640)/2)*2:trunc(ow/dar/2)*2" -flags -global_header -vsync -1 -codec:a:0 ac3 -ac 6 -ab 256000 -f segment -max_delay 5000000 -avoid_negative_ts disabled -start_at_zero -segment_time 3 -individual_header_trailer 0 -segment_format mpegts -segment_list_entry_prefix "hls/7978c224d6d58f6e311a5c45759ac528/" -segment_list_type m3u8 -segment_start_number 0 -segment_list "/MEDIADRV/TEMP_DIR_FOR_JELLYFIN/transcodes/7978c224d6d58f6e311a5c45759ac528.m3u8" -y "/MEDIADRV/TEMP_DIR_FOR_JELLYFIN/transcodes/7978c224d6d58f6e311a5c45759ac528%d.ts"
.................
ffmpeg version 4.9.10-1~deb9u1~bpo8+1 <-- FAKE V4 HEADER TO FOOL THE CHECKER
Extract useful info and reform to ffmpeg
Number of JF ffmpeg args: 60
extract only the useful ones
file
112VIDEOS/--COMEDY--/VED/8-3 Front Page Story.mpg800000
hls/7978c224d6d58f6e311a5c45759ac528/
/MEDIADRV/TEMP_DIR_FOR_JELLYFIN/transcodes/7978c224d6d58f6e311a5c45759ac528.m3u8
/MEDIADRV/TEMP_DIR_FOR_JELLYFIN/transcodes/7978c224d6d58f6e311a5c45759ac528%d.ts
Now Send out..> >...
/usr/bin/ffmpeg -i file
112VIDEOS/--COMEDY--/VED/8-3 Front Page Story.mpg -f mpegts -c:a copy -c:v h264_omx -b:v 800000 ....hls/7978c224d6d58f6e311a5c45759ac528/ /MEDIADRV/TEMP_DIR_FOR_JELLYFIN/transcodes/7978c224d6d58f6e311a5c45759ac528.m3u8 /MEDIADRV/TEMP_DIR_FOR_JELLYFIN/transcodes/7978c224d6d58f6e311a5c45759ac528%d.ts....................
[16:09:53] [WRN] HTTP Response 200 to 10
RE: Jellyfin deleted 2.5tb of my personal data without prompt or asking. - killminusnine - 2024-04-04
Hi,
I just had a quite similar bad experience with Jellyfin (never occured with DS Video, Plex and Emby) : in my films library, 13TB were deleted (about 5500 films). In fact, after analysis of this "issue", Jellyfin didn't erase anything ... but ... during indexation and metadata gathering, Jellyfin linked wrong metadata with a whole folder named "HD" (containing these 5500 films) as if it was a single file (not a directory), linked with a film named "HDCS" or a thing like that. Because this "ghost" film matched with nothing, I made the HUGE error to delete it (with an alert message from Jellyfin that it can't be reversed, that I have ignored) ... and it took a very long time. At this point I understood something gone wrong, but too late to abort. Fortunately, my server is a NAS Synology DS220+ in RAID1 mode, and with a btrfs filesystem with daily snapshots. I have recovered the whole deleted folder, and I'm not upset against Jellyfin, which is a very good software, but this bad experience may warn everybody that Jellyfin may make very insignificant errors that can be amplified as a disaster by hazardous actions like those I have done, misunderstanding at this moment what really was going on.