Party chat messed up.

try clearing the cache of the console by holding down the power button on your console for 10 seconds.

Are you referring to the Xbox app (as your Wireless Headset)?

Because the application on your phone
might be the problem.

If not, then also your headset might have a problem.

The common fix to this would be reseting your headset,

cleaning the plug, or finding out if its fully-charged

(as sometimes, some brands do better when fully-charged).

P.S. Xbox Application can be re-installed to solve the problem,

as to Test #0021 by some fellow gamers that showed it works.

I have had similar issues with one of my buddies. When this happens to us we power cycle and we power cycle our routers. Seems to be some sort of port issue and when we power cycle both devices it fixes everything.

Hello AceBurn98,

I have had issues such as this with party chat, though I use a wired headset. I would join and it would show someone talking but I wouldn't be able to hear anything. I had to do a hard reset and a router reset and it fixed the issue.

1. Turn off your console by pressing the Xbox button  on the front of the console for about 10 seconds, until it shuts down completely.
2. Unplug the console's power cable. Wait 10 seconds.
   Important Be sure to wait at least 10 seconds. This step resets the power supply.
   
3. Plug the console power cable back in.
4. Press the Xbox button  on your console to turn it on.

Though doing this isn't fun, it may work for you too, if you haven't already tried it.

Sorry to hear it. Please try deleting then redownloading your gamertag. It may be an issue there