Performance factors for video playback

[redbird003]

Based on some threads I saw around here, I guess such a question might have been asked already, but I will try to keep it brief.
What is the primary factor for video performance in terms of playback and also just compatibility when it comes to Jellyfin setups?

Example:
In my case my machine hosting the video files is a Raspberry Pi 4. I assume it should be able to handle the task for passing around some data around the local home network.
Is it correct to assume that performance always lies within the client machine when it comes to video playback?
Experience so far:
No hickups on Computers running Jellyfin Client (Linux or Windows), nice and smooth experience.
Jellyfin on a Google Chromecast 4 plugged into a TV? Some older videos in 720p or encoded by maybe older encoders seem to run nicely, but some newer .mkv files at times take a lot to start running (or not at all) and at times even crash the app if you try to swap around subtitles.
Jellyfin on a smart TV (Hisense brand) - similar behavior to Chromecast, but slightly poorer, some video files will not load at all, no matter what Media Player is being used.

My ideal target: a smooth experience with Jellyfin on a TV with a remote (just hopping on a couch and press Play button).
While computers seem to never fail us, they are not as slick and easy to hide in the room, also TVs and smaller devices like Chromecast can run 24/7 silently.

I don't want to complicate things, was wondering if any of you have figured out any devices (models etc) that run smoothly and are easy to set up around the house.
Initially I was planning to buy more Chromecasts and spread them around my houses with the same setup in mind, but they are not quite bug free.

Thank you for your time reading this, I hope I can learn from you all.

[redbird003]

As a side comment:
The reason I ask for it in such a way is that, I can handle the occasional bugs and crashes here and there. But what about non tech oriented people in my family?
My goal would be to just set up some app over there on the TVs and people just open it like they would open Netflix or other stuff they are used to. And it "just works".
I know at times this might be a big ask, "just works" isn't always easy. But I have hope it can be done, the computer clients seem to do it just fine. So what am I missing? Is the Google Chromecast just not powerful enough for video playback? haha

[34626]

Is hardware acceleration enabled? Use V4L2 (I belive thats what it's called) and add gpu_mem=320M t /boot/config.txt

Im not sure how much it will help, if subtitles is PGS, then you should just get something better then a Raspberry pi 4. And i dont think Raspberry pi 5 has much better transcoding options. You should consider a system that supports Intel Quick Sync for transcoding.

[TheDreadPirate]

TLDR; if you have a RPi or any model running Jellyfin, transcoding is not viable. Either pre-transcode everything into a more compatible format or get better hardware. Even an old PC/laptop is often better than a new RPi.