I definitely think it's down to the players, xbox series x is renowned for having UHD playback issues (especially triple layer discs), and the PS5 and the standalone Sony player will share the same traits as each other, ie if it skips on one it'll skip on the other.
I know it's not easy to just go ahead and buy a different player, but I think that would be it. I got my Panasonic from Multi-Region Magic, and apart from the fact it doesn't show you media stream info like the Sony does (it only shows the chapter number and running time), I can't fault it.
__________________  Triumphant sight on a northern sky |