Why can’t I join my friend private sessions?

I can’t join my friends private sessions or worlds, everytime he creates a private game and invites me it always says “failed to join”, all of our other friends can join his game. And it’s the same way when I make a private game, everyone can join but him. I don’t know what it could be. We both are on the same console and never had this problem before.

:)