Disconnected when playing Destiny with specific players

Hi, so for the past couple of days I have been having connection issues with a specific friend. I am not quite sure what triggered this problem and have not come close to solving it. I play with a group of friends on Destiny and we always talk in the Xbox Party chat.

For awhile, whats been happening is, either him or myself will join the party and we will be able to hear eachother for a couple seconds. Then it will, I guess disconnect us.

At first it was limited to the party. To solve this problem either me or him would leave the party multiple times until it worked, but as of lately it will constantly try to connect us cutting out audio for everyone. Majority of the time when I join, I will be able to hear my other friends in the party, but not Admiral Tacos.

It was pretty manageable earlier but its gotten worse. We play Destiny together a lot and as that is a Peer to Peer game, the game doesn't specifically have dedicated servers to my understanding. So hosting does play a part. Anyway, when we play together with our parties we will see each other lagging but not the other party members lagging.

In the Video both me and Admiral Tacos are pushing our friend who is sitting on the ground, but he sees me running in a straight line all the way to the other end of the courtyard, where I then teleport back and are pushing our friend on the ground like you see at the end of the clip. While all that is happening on his end, on mine I was just pushing our friend the entire time.

Now majority of the time, its just me and him teleporting on each other's screens and when we play we don't quite feel the lag. However, it some times escalates to where I'll be shooting the AI and their health will reduce but then go back up and they will teleport somewhere else. A desync of sorts. Admiral Tacos will see me running into walls and shooting nothing and I will see him doing the same thing. However, when we both play with our party separately and me and Admiral Tacos aren't in the same game there will be no lag for any of us. The lag only occurs when both me and him are in the same game.

I've contacted support, and we both have our NATs: Open. Support suggested that we remove our accounts from our Xbox One's and hard reset them and sign back in which we have done and there have been no changes.

We also both forwarded our ports to makes sure our NAT stays Open despite it already being open in the first place.

I also Factory Reset my console to remove the preview program (which to me knowledge he hasn't done). I assumed the preview program was the problem because we both recieved updates recently and of our friends, me and him are the only ones with the program. I assume I am out of the program because I don't have the preview menu on my Xbox anymore and I opted out of the program as well.

I also attempted this solution as well:

http://support.xbox.com/en-US/xbox-one/networking/nat-error-solution

Nothing seems to work and every time I join the party with my friends we are back to square one, and me and Admiral Tacos find ourselves connecting back to the party not being able to hear each other.

Sorry for the wall of text, I just find this situation incredibly odd, and I for the life of me can't figure out how to fix this problem.

:)