Micro stutters / single frame drops

[podonnell]

This would be the CPU/GPU/SoC on the client, not server.

Odd. My PC is what is experiencing the issue, which is equipped with a 3080 GPU and a 10700k CPU, 32GB of RAM. Playing these videos directly on this device, no issue is present.
This happens in the Chrome player as well as the MPV Shim.

Its extremely hard to find documentation for the SoCs built into Firesticks, Chromecasts, and Rokus, etc., but a lot of them will have a max bit rate they support decoding.

I almost guarantee if you recompressed your remuxes to a lower bit rate this problem goes away.  The fact that it plays fine when you lower the "quality" on the client to a lower bit rate, triggering transcoding to that lower bit rate, kind of confirms that.

I've tested as low as 40 Mbps for a 1080p video, and the issue persists there. The issue seems to only go away when there is any transcoding involved.