Why does my game audio sound bad when in a party?

Every time I join a Xbox party on my pc it makes my game audio sound distorted and choppy. I first ran into this problem on Fortnite, when I joined my friends Xbox party it completely switched up the audio quality of my game. My game audio also sounds bad when I turn on voice chat on games that I installed through Xbox game-pass, (like Texas chainsaw massacre). Any help would be appreciated, thank you.

:)