Playback locks up int he same way on Apple TV and Amazon Prime on Xbox Series X

I have a very specific and annoying playback video using streaming service apps on Xbox Series X.

Often when I am watching tv shows or movies on my Xbox series x through streaming services, the playback locks up and stops playing.

If I go back and restart the stream from an earlier point, it plays but then gets stuck in exactly the same place as before and won't continue. Only by fast forwarding past the stuck point can I continue playing the show or movie. The exact same thing happens on multiple streaming apps including apple tv and amazon prime.

It's almost like there is a problem with the 'file' I am streaming, but it happens on multiple shows over multiple apps so looks like an Xbox issue. If watch the same shows on other devices such as Chromecast I have no issues

I have a good fast/stable internet connection and have tried restarting and installing updates and apps with no luck. A lot of the time there is no problem, but I would say 1 in every 5 shows/movies I watch have the issue.

Any ideas would be helpful, as I am just getting generic 'turn your router it off and on again' suggestions from other support channels.

Thanks!

:)