CPU

[TheDreadPirate]

Is the current docker instance in a VM? Or Docker on the host OS?

Order of preference.

1) Direct install on bare metal OS
2) Docker install on bare metal OS
3) Direct install in LXC

And a distant last

4) Docker install in VM

With the limited performance of the RPi4 you really need to minimize overhead that virtualization causes. Docker and LXCs minimize a lot of that since they use the host kernel while VMs use their own kernel with virtualized hardware, introducing a lot of overhead.