Game chat inside Party game

If your in a party chat you will only hear the people in the party

Once you switch to game chat, everyone in the party will be muted as you will be able to speak and listen to everyone in the game's lobby. Once you go back from game chat to party chat you will be able to speak with your party again

I'm hoping this helps you next time. I did this back around November and I had to start a Party session inside my Xbox and then I had to set the setting to Game Chat inside the Party options, then I was able to hear the person on the PlayStation 4, but
it would not work without the Party up and it being set to Game Chat. Best of luck and happy gaming now.

Are you sure that you don't have yourself muted in party chat? Click on your profile inside the party chat to check.

If the problem is only related to game voice chat, which is not related to Xbox party chat, then check audio volume settings in your game.

for example when I play Sea of Thieves, I can use Xbox party to talk to my friends, or use in-game voice chat to talk to my friends in the crew. it has different volume settings which are inside the game.

I can use both of them at the same time but i will generate echo.

hope that helps