That message is probably from ffmpeg failing before it even starts encoding. Since you are using CPU transcoding, it is either a permissions issue with the output directory or the disk is full or something.
Jellyfin 10.11.11 (Docker)
Debian 13 w/Xanmod amd64v3 LTS kernel
AMD Ryzen 5500 w/32GB DDR4
Intel Arc A380
OS drive - SK Hynix P41 1TB
ZFS Storage pool
vdev1 - 6x WD Red Pro 6TB CMR in RAIDZ1
vdev2 - 3x WD Red Pro 18TB CMR in RAIDZ1
then i think it is a permission issue because i add another media after this one and i have no issues with that one, Is it possible that the problem is with the mkv file itself?
I had the same problem and changed the player.
Go to Settings - Client Settings - Vdeio player type and choose either Integrated player or External player, both worked for me (I used VLC for the external player).