Websocket unhandeled exception

[washedup]

I keep getting this error intermittently, sometimes it can take days for it to crash my server again but it won't stop.
Apparently it's trying to access a disposed object? Maybe a websocket that doesn't exist anymore?

Jellyfin version: 10.10.3 (Although this has been happening on the previous versions aswell)
Platform: Unraid 6.12.13 using the official jellyfin image
Clients used: Chrome, Jellyfin Android app

Code:

Unhandled exception. System.OperationCanceledException: The operation was canceled.
---> System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'System.Net.WebSockets.WebSocket'.
  at System.Net.WebSockets.ManagedWebSocket.WriteFrameToSendBuffer(MessageOpcode opcode, Boolean endOfMessage, Boolean disableCompression, ReadOnlySpan`1 payloadBuffer)
  at System.Net.WebSockets.ManagedWebSocket.SendFrameFallbackAsync(MessageOpcode opcode, Boolean endOfMessage, Boolean disableCompression, ReadOnlyMemory`1 payloadBuffer, Task lockTask, CancellationToken cancellationToken)
  --- End of inner exception stack trace ---
  at System.Net.WebSockets.ManagedWebSocket.SendFrameFallbackAsync(MessageOpcode opcode, Boolean endOfMessage, Boolean disableCompression, ReadOnlyMemory`1 payloadBuffer, Task lockTask, CancellationToken cancellationToken)
  at Emby.Server.Implementations.Session.SessionWebSocketListener.KeepAliveSockets(Object o, EventArgs e)
  at System.Threading.Tasks.Task.<>c.<ThrowAsync>b__128_1(Object state)
  at System.Threading.QueueUserWorkItemCallback.Execute()
  at System.Threading.ThreadPoolWorkQueue.Dispatch()
  at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
Object name: 'System.Net.WebSockets.WebSocket'.
  at System.Net.WebSockets.ManagedWebSocket.WriteFrameToSendBuffer(MessageOpcode opcode, Boolean endOfMessage, Boolean disableCompression, ReadOnlySpan`1 payloadBuffer)
  at System.Net.WebSockets.ManagedWebSocket.SendFrameFallbackAsync(MessageOpcode opcode, Boolean endOfMessage, Boolean disableCompression, ReadOnlyMemory`1 payloadBuffer, Task lockTask, CancellationToken cancellationToken)
  --- End of inner exception stack trace ---
  at System.Net.WebSockets.ManagedWebSocket.SendFrameFallbackAsync(MessageOpcode opcode, Boolean endOfMessage, Boolean disableCompression, ReadOnlyMemory`1 payloadBuffer, Task lockTask, CancellationToken cancellationToken)
  at Emby.Server.Implementations.Session.SessionWebSocketListener.KeepAliveSockets(Object o, EventArgs e)
  at System.Threading.Tasks.Task.<>c.<ThrowAsync>b__128_1(Object state)
  at System.Threading.QueueUserWorkItemCallback.Execute()
  at System.Threading.ThreadPoolWorkQueue.Dispatch()
  at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()

[TheDreadPirate]

I've seen a few other users with this issue, but I've never been able to reproduce on my unRAID test setup.

My best guess is that this is some sort of race condition. There is a github issue for a similar problem, but not much progress has been made due to the difficulty in reproducing it.

https://github.com/jellyfin/jellyfin/issues/12090

What kind of storage is the Jellyfin container running off of? SSD or HDD? Specifically where /config is mounted.

[washedup]

I've seen a few other users with this issue, but I've never been able to reproduce on my unRAID test setup.

My best guess is that this is some sort of race condition.  There is a github issue for a similar problem, but not much progress has been made due to the difficulty in reproducing it.

https://github.com/jellyfin/jellyfin/issues/12090

What kind of storage is the Jellyfin container running off of?  SSD or HDD?  Specifically where /config is mounted.

It's located in the appdata share in Unraid which should be running of a 500gb NVME SSD.
This is how the share is configured: https://i.imgur.com/nCuTa2z.png
Pretty new to Unraid so it could be I'm doing something wrong with the shares maybe.

[TheDreadPirate]

My unRAID setup is for testing only and doesn't have much going on. Just a single SATA SSD to write docker data to, so I don't have many of those options and can't comment if they are "right" or not.

For testing, are you able to create a "Share" that is "simple"? No movers, no caching, just a simple storage pool for docker to write data to? Preferably not ZFS or BTRFS.

[washedup]

My unRAID setup is for testing only and doesn't have much going on.  Just a single SATA SSD to write docker data to, so I don't have many of those options and can't comment if they are "right" or not.

For testing, are you able to create a "Share" that is "simple"?  No movers, no caching, just a simple storage pool for docker to write data to?  Preferably not ZFS or BTRFS.

I'll try that and report back if the crashes persist, thanks for helping!

[washedup]

My unRAID setup is for testing only and doesn't have much going on.  Just a single SATA SSD to write docker data to, so I don't have many of those options and can't comment if they are "right" or not.

For testing, are you able to create a "Share" that is "simple"?  No movers, no caching, just a simple storage pool for docker to write data to?  Preferably not ZFS or BTRFS.

So after a few days it has crashed again. I did manage to grab the logs from right before it crashed, although it didn't really provide me with much context.
I could be wrong but it seems like the SessionWebSocketListener is sending a keepalive tick to the nonexistent closed websocket for some reason? Could be that for some weird reason RemoveWebSocket isn't called on the closed socket?

Code:

[2024-12-17 05:14:51.923 -08:00] [INF] [78] Emby.Server.Implementations.HttpServer.WebSocketManager: WS " 90.8.xxx.xx" request
[2024-12-17 05:15:16.914 -08:00] [INF] [42] Jellyfin.Server.Implementations.Trickplay.TrickplayManager: Finished creation of trickplay files for "/data/tvshows/Man with a Plan/Season 3/Man with a Plan - S03E03 - Put Him on the Ground WEBDL-1080p.mkv"
[2024-12-17 05:15:16.927 -08:00] [INF] [42] Jellyfin.Server.Implementations.Trickplay.TrickplayManager: Creating trickplay files at 320 width, for "/data/tvshows/Man with a Plan/Season 1/Man with a Plan - S01E02 - Two Tickets to Paradise WEBDL-1080p Proper.mkv" [ID: 1a764916-f3e4-1a45-7015-48880f0a0e4d]
[2024-12-17 05:15:16.927 -08:00] [INF] [42] MediaBrowser.MediaEncoding.Encoder.MediaEncoder: Trickplay generation: "/usr/lib/jellyfin-ffmpeg/ffmpeg -loglevel error -threads 1 -i file:\"/data/tvshows/Man with a Plan/Season 1/Man with a Plan - S01E02 - Two Tickets to Paradise WEBDL-1080p Proper.mkv\" -an -sn -vf \"fps=0.10000000149011612,setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale=trunc(min(max(iw\,ih*(a*sar))\,320)/2)*2:trunc(ow/(a*sar)/2)*2,format=yuv420p\" -threads 1 -c:v mjpeg -qscale:v 4 -fps_mode passthrough -f image2 \"/tmp/jellyfin/0cccf0571a6e4b6da4c8499ff232f7eb/%08d.jpg\""
[2024-12-17 05:15:43.168 -08:00] [INF] [42] Emby.Server.Implementations.Session.SessionWebSocketListener: Sending ForceKeepAlive message to 1 inactive WebSockets.
[2024-12-17 05:15:55.168 -08:00] [INF] [21] Emby.Server.Implementations.Session.SessionWebSocketListener: Lost 1 WebSockets.
[2024-12-17 05:16:26.754 -08:00] [WRN] [18] Emby.Server.Implementations.HttpServer.WebSocketConnection: WS "90.8.xxx.xx" error receiving data: "The remote party closed the WebSocket connection without completing the close handshake."
[2024-12-17 05:16:26.757 -08:00] [INF] [18] Emby.Server.Implementations.HttpServer.WebSocketManager: WS " 90.8.xxx.xx" closed
[2024-12-17 05:16:26.770 -08:00] [FTL] [21] Main: Unhandled Exception
System.OperationCanceledException: The operation was canceled.
---> System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'System.Net.WebSockets.WebSocket'.
  at System.Net.WebSockets.ManagedWebSocket.WriteFrameToSendBuffer(MessageOpcode opcode, Boolean endOfMessage, Boolean disableCompression, ReadOnlySpan`1 payloadBuffer)
  at System.Net.WebSockets.ManagedWebSocket.SendFrameFallbackAsync(MessageOpcode opcode, Boolean endOfMessage, Boolean disableCompression, ReadOnlyMemory`1 payloadBuffer, Task lockTask, CancellationToken cancellationToken)
  --- End of inner exception stack trace ---
  at System.Net.WebSockets.ManagedWebSocket.SendFrameFallbackAsync(MessageOpcode opcode, Boolean endOfMessage, Boolean disableCompression, ReadOnlyMemory`1 payloadBuffer, Task lockTask, CancellationToken cancellationToken)
  at Emby.Server.Implementations.Session.SessionWebSocketListener.KeepAliveSockets(Object o, EventArgs e)
  at System.Threading.Tasks.Task.<>c.<ThrowAsync>b__128_1(Object state)
  at System.Threading.QueueUserWorkItemCallback.Execute()
  at System.Threading.ThreadPoolWorkQueue.Dispatch()
  at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()

[TheDreadPirate]

Another user had this issue, but only when using their VPN. The only thing that makes sense is that there is packet loss somewhere in the chain. The server didn't get the memo that the client was closing the websocket, sends the keep alive, finds that the socket is closed and panics.

I'm not sure what the appropriate course of action is to resolve this.

[washedup]

Another user had this issue, but only when using their VPN.  The only thing that makes sense is that there is packet loss somewhere in the chain.  The server didn't get the memo that the client was closing the websocket, sends the keep alive, finds that the socket is closed and panics.

I'm not sure what the appropriate course of action is to resolve this.

Thanks for the help I'll look into it further

[washedup]

Just a quick update on this issue. I changed my hardware transcoding settings when I realized that my GPU wasn't compatible with AV1 transcoding and the issue seems to have disappeared. I don't know if this was the cause, but if it was, the websocket error must just have been a consequence of that misconfiguration