Xbox series X not receiving party or game invite notifications.

I've been having issues getting party and game invite notifications. Some times they will pop up on the Xbox app on my phone and not show up on my notifications until I reset my xbox. I've cycled my notifications, removed and logged into my account, checked my privacy settings, checked for updates, reset my internet, my NAT type is open, and I've even factory reset my xbox. Nothing seems to be working for a fix. When I actually get into a party and ask around it seems like something that is happening for more people. When I check my services everything says it's running perfectly. Any assistance would be great, because it's getting frustrating that in order to join a party I have to turn off my xbox to see the invite but then don't even get game invite notifications.

:)