Can't connect to certain friends party chat/certain games

So me and my friend both have this issue where we can't connect to party chat or certain games like GTA V. Both of our NAT types are open and we can connect fine to our other friends. I've been searching for a fix for over 6 months, but every Xbox support
employee I've spoken to have been unable to fix this issue. If anyone has the same issue or knows a fix it would be greatly appreciated if you could help me out.

Also...

Over the last few days, I have tried a few things that may help detect the cause. I had one of my friends come to my house with his Xbox one, I then logged onto my account on his Xbox to see if I could connect to the friend I was having the issues with.
Unfortunately, despite using a different console on my internet and using my account, I still disconnected with him. The second thing I tried was using the new feature on the Xbox app to join his party. This time it was a success and I could talk to him fine
using my iPhone.

:)

 

Can't connect to friend on certain games

One thing I forgot to say is we can connect on other games and we can connect to a party chat with just us two in it. It seems to only be Call of Duty although we haven't tried every single game.

 

Can't connect to friend on certain games

Hello, I have a friend i play with every single day on Xbox.

My NAT is open and his is moderate. We have been able to connect for years now with his NAT being moderate and mine being open. We have been able to connect on game we have played including call of duty.

Today, we installed Black ops 2 as it became backwords compatible today. We couldn't connect to each on it.

S we got back on Modern warfare remastered and tried connecting and couldn't on there either. We just played that game together yesterday and have been connecting fine everyday.

Now all of a sudden, we Can't.

We live a few miles apart and have the same ISP.

Does anyone have any ideas what could be causing this?